Sep 11 16:15:11.937957 ip-10-0-136-18 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 16:15:11.937971 ip-10-0-136-18 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 16:15:11.937981 ip-10-0-136-18 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 16:15:11.938253 ip-10-0-136-18 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 16:15:22.131924 ip-10-0-136-18 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 16:15:22.131946 ip-10-0-136-18 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a3b65366ad59499fb5c010ff0e4d4287 --
Sep 11 16:17:22.504084 ip-10-0-136-18 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 16:17:22.913555 ip-10-0-136-18 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:22.913555 ip-10-0-136-18 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 16:17:22.913555 ip-10-0-136-18 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:22.913555 ip-10-0-136-18 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 16:17:22.913555 ip-10-0-136-18 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 16:17:22.914249 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.913598    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 16:17:22.917595 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917579    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.917595 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917595    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917599    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917602    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917606    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917609    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917611    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917614    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917617    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917620    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917623    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917625    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917628    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917631    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917647    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917650    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917653    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917655    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917658    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917661    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.917656 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917664    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917667    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917670    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917672    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917675    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917678    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917680    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917683    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917686    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917688    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917691    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917693    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917698    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917702    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917705    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917708    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917710    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917731    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917735    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917738    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.918122 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917741    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917744    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917746    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917761    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917764    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917766    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917769    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917772    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917775    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917778    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917781    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917784    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917788    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917792    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917795    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917798    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917801    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917805    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917807    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.918645 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917810    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917813    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917816    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917818    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917823    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917827    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917830    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917832    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917835    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917838    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917840    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917843    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917846    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917848    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917850    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917853    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917855    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917858    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917860    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.919376 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917863    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917866    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917869    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917872    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917875    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917877    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917879    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917882    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917885    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917888    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917891    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917893    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917896    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917898    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917901    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917903    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917906    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917908    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917911    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917914    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.919905 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.917916    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.920478 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920461    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.920478 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920477    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920480    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920484    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920487    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920489    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920494    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920497    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920499    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920502    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920505    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920507    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920511    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920513    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920516    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920519    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920521    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920524    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920526    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920529    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920532    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.920534 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920534    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920537    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920540    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920543    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920547    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920550    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920552    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920555    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920557    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920560    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920563    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920565    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920568    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920570    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920573    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920576    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920578    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920581    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920583    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920585    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.921080 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920588    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920591    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920595    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920602    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920606    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920609    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920614    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920616    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920619    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920623    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920626    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920628    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920631    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920633    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920637    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920640    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920642    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920645    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920648    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.921561 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920650    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920653    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920655    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920658    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920661    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920665    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920669    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920671    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920674    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920677    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920679    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920682    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920684    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920687    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920691    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920693    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920696    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920698    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920701    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920704    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.922077 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920706    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920708    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920711    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920714    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920718    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920722    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920725    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920728    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920731    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920733    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920736    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920739    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920741    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920744    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920746    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920761    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920764    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920767    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.920770    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.922566 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921164    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921169    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921173    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921177    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921181    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921184    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921187    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921190    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921193    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921196    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921199    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921201    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921205    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921207    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921210    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921213    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921216    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921219    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921221    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921224    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.923048 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921228    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921230    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921233    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921235    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921238    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921240    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921243    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921245    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921247    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921251    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921253    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921256    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921258    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921261    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921264    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921267    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921270    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921272    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921274    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921277    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.923531 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921279    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921282    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921284    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921286    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921289    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921291    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921295    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921299    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921302    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921305    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921307    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921310    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921313    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921316    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921319    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921321    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921324    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921326    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921329    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.924027 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921331    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921334    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921337    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921339    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921343    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921346    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921348    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921351    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921353    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921356    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921358    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921361    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921363    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921366    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921368    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921371    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921373    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921376    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921378    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.924477 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921381    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921383    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921385    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921388    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921391    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921394    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921396    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921399    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921401    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921404    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921406    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921409    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921411    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921413    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921416    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921419    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921422    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921424    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921426    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921429    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.924957 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.921432    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921512    2579 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921548    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921555    2579 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921559    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921564    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921567    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921571    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921584    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921587    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921591    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921594    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921597    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921600    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921603    2579 flags.go:64] FLAG: --cgroup-root=""
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921606    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921608    2579 flags.go:64] FLAG: --client-ca-file=""
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921612    2579 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921615    2579 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921619    2579 flags.go:64] FLAG: --cluster-domain=""
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921622    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921625    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921628    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921631    2579 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 16:17:22.925440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921635    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921637    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921641    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921644    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921647    2579 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921650    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921653    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921656    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921659    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921663    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921666    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921668    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921671    2579 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921674    2579 flags.go:64] FLAG: --enable-server="true"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921677    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921681    2579 flags.go:64] FLAG: --event-burst="100"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921684    2579 flags.go:64] FLAG: --event-qps="50"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921687    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921690    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921692    2579 flags.go:64] FLAG: --eviction-hard=""
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921696    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921699    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921702    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921705    2579 flags.go:64] FLAG: --eviction-soft=""
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921707    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 16:17:22.926030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921713    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921716    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921719    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921722    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921725    2579 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921728    2579 flags.go:64] FLAG: --feature-gates=""
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921732    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921735    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921740    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921743    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921746    2579 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921762    2579 flags.go:64] FLAG: --help="false"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921765    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-136-18.ec2.internal"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921768    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921772    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921775    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921778    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921781    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921784    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921787    2579 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921790    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921793    2579 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921796    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921799    2579 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 16:17:22.926695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921801    2579 flags.go:64] FLAG: --kube-reserved=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921804    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921807    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921810    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921813    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921816    2579 flags.go:64] FLAG: --lock-file=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921818    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921821    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921825    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921830    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921834    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921837    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921840    2579 flags.go:64] FLAG: --logging-format="text"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921843    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921846    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921849    2579 flags.go:64] FLAG: --manifest-url=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921853    2579 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921858    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921861    2579 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921866    2579 flags.go:64] FLAG: --max-pods="110"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921868    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921872    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921874    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921877    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921880    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 16:17:22.927316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921883    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921886    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921893    2579 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921896    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921899    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921902    2579 flags.go:64] FLAG: --pod-cidr=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921905    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921911    2579 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921914    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921917    2579 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921920    2579 flags.go:64] FLAG: --port="10250"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921923    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921926    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08e2fd2e2301814b7"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921929    2579 flags.go:64] FLAG: --qos-reserved=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921932    2579 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921935    2579 flags.go:64] FLAG: --register-node="true"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921939    2579 flags.go:64] FLAG: --register-with-taints=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921943    2579 flags.go:64] FLAG: --registry-burst="10"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921949    2579 flags.go:64] FLAG: --registry-qps="5"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921951    2579 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921954    2579 flags.go:64] FLAG: --reserved-memory=""
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921958    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921961    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921965    2579 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921968    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 16:17:22.927936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921971    2579 flags.go:64] FLAG: --runonce="false"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921974    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921977    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921980    2579 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921983    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921986    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921989    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921992    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921995    2579 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.921998    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922000    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922003    2579 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922006    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922009    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922012    2579 flags.go:64] FLAG: --system-cgroups=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922014    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922019    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922022    2579 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922025    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922029    2579 flags.go:64] FLAG: --tls-min-version=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922032    2579 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922035    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922037    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922041    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922044    2579 flags.go:64] FLAG: --v="2"
Sep 11 16:17:22.928546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922048    2579 flags.go:64] FLAG: --version="false"
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922052    2579 flags.go:64] FLAG: --vmodule=""
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922056    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922059    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922185    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922191    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922194    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922196    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922199    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922202    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922205    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922207    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922210    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922213    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922216    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922219    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922221    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922224    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922226    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922229    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.929202 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922231    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922234    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922236    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922239    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922242    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922244    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922247    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922250    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922252    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922257    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922261    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922265    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922267    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922270    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922273    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922276    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922278    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922282    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922285    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922287    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.929691 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922290    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922293    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922295    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922298    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922301    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922303    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922306    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922308    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922311    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922314    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922317    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922319    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922322    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922325    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922327    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922338    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922341    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922344    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922346    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922349    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.930239 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922351    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922355    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922360    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922363    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922365    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922368    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922371    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922374    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922376    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922380    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922383    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922385    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922388    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922390    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922393    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922396    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922398    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922401    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922403    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.930735 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922406    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922408    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922411    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922413    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922416    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922418    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922421    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922424    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922426    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922429    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922431    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922434    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922436    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922439    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922441    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922445    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922448    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922450    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922454    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.931231 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922458    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922461    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922464    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922468    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.922472    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.922954    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.930079    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.930095    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.930917    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.930926    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.930986    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.930991    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.930995    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.930998    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931002    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.931699 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931005    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931007    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931010    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931013    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931016    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931018    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931021    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931023    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931028    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931032    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931035    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931038    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931041    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931044    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931046    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931049    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931052    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931054    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931057    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.932086 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931064    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931067    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931070    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931073    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931075    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931079    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931082    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931085    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931088    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931090    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931093    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931096    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931098    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931101    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931103    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931106    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931109    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931111    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931114    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931117    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.932550 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931119    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931122    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931124    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931127    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931130    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931132    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931135    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931138    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931141    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931143    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931146    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931148    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931151    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931154    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931157    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931159    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931161    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931165    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931167    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931170    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.933056 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931173    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931175    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931178    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931180    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931183    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931185    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931188    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931190    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931194    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931198    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931201    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931204    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931206    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931209    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931211    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931214    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931216    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931219    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931222    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931224    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.933549 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931227    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931230    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931232    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931235    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931238    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931240    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931244    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931247    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931249    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931254    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931258    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931261    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931263    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931266    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.934153 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931269    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.931274    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931385    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931389    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931392    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931395    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931398    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931401    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931404    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931407    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931410    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931413    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931415    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931419    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931423    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 16:17:22.934502 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931426    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931429    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931432    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931435    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931438    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931441    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931445    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931448    2579 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931450    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931453    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931456    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931459    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931462    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931465    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931468    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931470    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931473    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931477    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931480    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 16:17:22.934982 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931483    2579 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931486    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931489    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931492    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931495    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931497    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931500    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931503    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931505    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931508    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931511    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931513    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931516    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931518    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931521    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931524    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931527    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931530    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931532    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931535    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 16:17:22.935450 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931537    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931540    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931542    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931545    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931548    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931551    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931554    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931556    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931559    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931561    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931564    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931566    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931569    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931571    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931574    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931577    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931579    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931581    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931584    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931586    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 16:17:22.935948 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931589    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931591    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931594    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931597    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931599    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931602    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931604    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931606    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931609    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931611    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931614    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931617    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931619    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931622    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931624    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931627    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931629    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931632    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931634    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 16:17:22.936428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931637    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931640    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931642    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931645    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931647    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931650    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931652    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:22.931655    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.931660    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.931782    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.933828    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.934659    2579 server.go:1019] "Starting client certificate rotation"
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.934767    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:17:22.936902 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.934800    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 16:17:22.937250 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.937212    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:17:22.938408 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.938391    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 16:17:22.949900 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.949883    2579 log.go:25] "Validated CRI v1 runtime API"
Sep 11 16:17:22.952806 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.952788    2579 log.go:25] "Validated CRI v1 image API"
Sep 11 16:17:22.953671 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.953653    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 16:17:22.959083 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.959058    2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8ca9cbcd-7ef2-471f-a3f1-8b76b3b66b99:/dev/nvme0n1p4 c75b4145-3b18-4185-a7b9-e36bc8275fe6:/dev/nvme0n1p3]
Sep 11 16:17:22.959139 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.959081    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 16:17:22.964185 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.964169    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:17:22.965506 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.965395    2579 manager.go:216] Machine: {Timestamp:2026-09-11 16:17:22.964008384 +0000 UTC m=+0.343944212 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102391 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c49d4f8adb1044061136127270502 SystemUUID:ec2c49d4-f8ad-b104-4061-136127270502 BootID:a3b65366-ad59-499f-b5c0-10ff0e4d4287 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:3d:f2:50:91:a9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:3d:f2:50:91:a9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:35:3e:b2:e7:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 16:17:22.965506 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.965506    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 16:17:22.965612 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.965591    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 16:17:22.967406 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.967385    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 16:17:22.967543 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.967411    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-18.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 16:17:22.968162 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.968152    2579 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 16:17:22.968193 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.968165    2579 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 16:17:22.968193 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.968173    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:17:22.968783 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.968774    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 16:17:22.969553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.969543    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 16:17:22.973240 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973229    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:17:22.973361 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973352    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 16:17:22.973395 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973382    2579 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 16:17:22.973395 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973392    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 16:17:22.973447 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973407    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 16:17:22.973447 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973416    2579 kubelet.go:398] "Adding apiserver pod source"
Sep 11 16:17:22.973447 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973425    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 16:17:22.973853 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973841    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:17:22.973887 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.973863    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 16:17:22.976358 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.976337    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 16:17:22.978579 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.978566    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 16:17:22.978634 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.978596    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 16:17:22.980081 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980069    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980087    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980094    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980100    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980109    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980115    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 16:17:22.980127 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980129    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 16:17:22.980277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980135    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 16:17:22.980277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980142    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 16:17:22.980277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980148    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 16:17:22.980277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980165    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 16:17:22.980277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980171    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 16:17:22.980951 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980942    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 16:17:22.980986 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.980952    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 16:17:22.982383 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:22.982355    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 16:17:22.982727 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:22.982706    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 16:17:22.984896 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.984883    2579 server.go:1268] "Started kubelet"
Sep 11 16:17:22.984985 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.984954    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 16:17:22.985078 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.984961    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 16:17:22.985078 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.985033    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 16:17:22.985941 ip-10-0-136-18 systemd[1]: Started Kubernetes Kubelet.
Sep 11 16:17:22.986053 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.985980    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 16:17:22.991436 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:22.991415    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 16:17:22.991630 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.991617    2579 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 16:17:22.994422 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.994403    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 16:17:22.994920 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.994902    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 16:17:22.995046 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:22.995030    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:22.995735 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995716    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 16:17:22.995735 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995720    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 16:17:22.995872 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995745    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 16:17:22.995924 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995873    2579 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 16:17:22.995924 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995880    2579 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 16:17:22.995924 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995912    2579 factory.go:153] Registering CRI-O factory
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995940    2579 factory.go:223] Registration of the crio container factory successfully
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995988    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5dgp5"
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.995993    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.996004    2579 factory.go:55] Registering systemd factory
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.996011    2579 factory.go:223] Registration of the systemd container factory successfully
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.996025    2579 factory.go:103] Registering Raw factory
Sep 11 16:17:22.996034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.996034    2579 manager.go:1196] Started watching for new ooms in manager
Sep 11 16:17:22.996304 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.996295    2579 manager.go:349] Starting recovery of all containers
Sep 11 16:17:22.997099 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:22.997073    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-18.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 16:17:22.998289 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:22.997174    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-18.ec2.internal.18d4507b6b54df22  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-18.ec2.internal,UID:ip-10-0-136-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-18.ec2.internal,},FirstTimestamp:2026-09-11 16:17:22.984853282 +0000 UTC m=+0.364789105,LastTimestamp:2026-09-11 16:17:22.984853282 +0000 UTC m=+0.364789105,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-18.ec2.internal,}"
Sep 11 16:17:23.000846 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.000643    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 16:17:23.000846 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.000772    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 16:17:23.004884 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.004719    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5dgp5"
Sep 11 16:17:23.005674 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.005661    2579 manager.go:354] Recovery completed
Sep 11 16:17:23.009933 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.009920    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.012354 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.012337    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.012426 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.012370    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.012426 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.012384    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.013013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013001    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 16:17:23.013013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013012    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 16:17:23.013013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013014    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.013218 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013026    2579 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 16:17:23.013809 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013796    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.013857 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013822    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.013857 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.013835    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.014292 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.014279    2579 policy_none.go:49] "None policy: Start"
Sep 11 16:17:23.014349 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.014298    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 16:17:23.014349 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.014309    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 16:17:23.014579 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.014510    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-18.ec2.internal.18d4507b6cf87fcf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-18.ec2.internal,UID:ip-10-0-136-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-18.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-18.ec2.internal,},FirstTimestamp:2026-09-11 16:17:23.012353999 +0000 UTC m=+0.392289824,LastTimestamp:2026-09-11 16:17:23.012353999 +0000 UTC m=+0.392289824,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-18.ec2.internal,}"
Sep 11 16:17:23.015678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.015667    2579 policy_none.go:47] "Start"
Sep 11 16:17:23.050667 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.050650    2579 manager.go:337] "Starting Device Plugin manager"
Sep 11 16:17:23.050782 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.050691    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 16:17:23.050782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.050704    2579 server.go:87] "Starting device plugin registration server"
Sep 11 16:17:23.051070 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.051057    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 16:17:23.051107 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.051073    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 16:17:23.051184 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.051164    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 16:17:23.051271 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.051252    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 16:17:23.051271 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.051267    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 16:17:23.052974 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.052838    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 16:17:23.052974 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.052891    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.136379 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.136334    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 16:17:23.137716 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.137660    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 16:17:23.137716 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.137707    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 16:17:23.137875 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.137736    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 16:17:23.137875 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.137801    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 16:17:23.140860 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.140840    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:23.151523 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.151506    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.152311 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.152296    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.152397 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.152329    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.152397 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.152343    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.152397 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.152374    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.161391 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.161371    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.161461 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.161393    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-18.ec2.internal\": node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.178770 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.178682    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.238609 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.238579    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"]
Sep 11 16:17:23.238772 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.238666    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.240317 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.240299    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.240387 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.240337    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.240387 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.240352    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.241654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.241642    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.241791 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.241774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.241846 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.241805    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.242401 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242388    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.242401 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242394    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.242494 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242411    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.242494 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242415    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.242494 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242422    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.242494 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.242426    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.243635 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.243623    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.243680 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.243649    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 16:17:23.244313 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.244299    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 16:17:23.244377 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.244325    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 16:17:23.244377 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.244335    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 16:17:23.268376 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.268354    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-18.ec2.internal\" not found" node="ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.272580 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.272565    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-18.ec2.internal\" not found" node="ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.279614 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.279600    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.297913 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.297891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.297983 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.297918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bdb01b23407b83abb142f3983543d1-config\") pod \"kube-apiserver-proxy-ip-10-0-136-18.ec2.internal\" (UID: \"34bdb01b23407b83abb142f3983543d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.297983 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.297934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.379938 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.379912    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.398125 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398094    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.398125 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398110    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.398299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.398299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bdb01b23407b83abb142f3983543d1-config\") pod \"kube-apiserver-proxy-ip-10-0-136-18.ec2.internal\" (UID: \"34bdb01b23407b83abb142f3983543d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.398299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398207    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/34bdb01b23407b83abb142f3983543d1-config\") pod \"kube-apiserver-proxy-ip-10-0-136-18.ec2.internal\" (UID: \"34bdb01b23407b83abb142f3983543d1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.398299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.398247    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a631e1211894d68a5cbaa9874d6ed29-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal\" (UID: \"9a631e1211894d68a5cbaa9874d6ed29\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.481012 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.480928    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.571448 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.571422    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.575693 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.575406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.581148 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.581133    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.681774 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.681714    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.782262 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.782193    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-18.ec2.internal\" not found"
Sep 11 16:17:23.853558 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.853530    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:23.895377 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.895349    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.907624 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.907593    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:17:23.909588 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.909571    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal"
Sep 11 16:17:23.919544 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.919527    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 16:17:23.934716 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.934693    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 16:17:23.934877 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.934854    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:17:23.934934 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.934881    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 16:17:23.974013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.973986    2579 apiserver.go:52] "Watching apiserver"
Sep 11 16:17:23.989839 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.989816    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 16:17:23.990221 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.990199    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-68vlf","openshift-multus/multus-sqwzq","openshift-network-diagnostics/network-check-target-xp82l","openshift-ovn-kubernetes/ovnkube-node-9ctcb","kube-system/global-pull-secret-syncer-ft8q5","openshift-cluster-node-tuning-operator/tuned-k6zb9","openshift-image-registry/node-ca-zjzhz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal","openshift-multus/network-metrics-daemon-r6hs9","openshift-network-operator/iptables-alerter-wsddv","kube-system/konnectivity-agent-tcclt","kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj","openshift-dns/node-resolver-knxgq"]
Sep 11 16:17:23.991971 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.991950    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:23.993171 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.993151    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:23.994525 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994506    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 16:17:23.994766 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994719    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:23.994766 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994731    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:23.994766 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994720    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-plqzg\""
Sep 11 16:17:23.994940 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994772    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:23.994940 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:23.994806    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:23.994940 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.994731    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 16:17:23.996861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.996150    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 16:17:23.996861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.996443    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pvbm2\""
Sep 11 16:17:23.996861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.996649    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 16:17:23.998115 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.998093    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:23.999536 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:23.999512    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:24.000087 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.000066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.000184 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.000160    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.000245 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.000230    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mj2s9\""
Sep 11 16:17:24.000958 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.000944    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:24.001042 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.000985    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a373d3fc-9c59-4ea6-841b-fc50591c663e-iptables-alerter-script\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:24.001042 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-579bq\" (UniqueName: \"kubernetes.io/projected/a373d3fc-9c59-4ea6-841b-fc50591c663e-kube-api-access-579bq\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:24.001042 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001023    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-agent-certs\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:24.001195 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4bbaa63-55aa-4948-8f3c-587253f132af-hosts-file\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:24.001195 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a373d3fc-9c59-4ea6-841b-fc50591c663e-host-slash\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:24.001195 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-konnectivity-ca\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:24.001195 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:24.001354 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bbaa63-55aa-4948-8f3c-587253f132af-tmp-dir\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:24.001354 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rc6t\" (UniqueName: \"kubernetes.io/projected/a4bbaa63-55aa-4948-8f3c-587253f132af-kube-api-access-2rc6t\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:24.001962 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001940    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.001962 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.001957    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.002119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.002055    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 16:17:24.002119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.002091    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fh5bs\""
Sep 11 16:17:24.002545 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.002527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:24.002626 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:24.002604    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:24.003403 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.003382    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.003575 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.003560    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dd5v6\""
Sep 11 16:17:24.003778 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.003705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.004032 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.004018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:24.005451 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.005432    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:24.006544 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.006322    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.006638 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.006564    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 16:17:24.007246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.007222    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:24.007350 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.007330    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h5fcc\""
Sep 11 16:17:24.007440 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.007420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.007937    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008035    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wxpqb\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008135    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008194    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008244    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008839    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 16:12:24 +0000 UTC" deadline="2028-02-16 15:15:46.959868513 +0000 UTC"
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008875    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12550h58m22.950996028s"
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.008956    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:24.009028    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.009543    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xspvg\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.009578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 16:17:24.009677 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.009578    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 16:17:24.010315 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.009986    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 16:17:24.010459 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.010442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:24.013028 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.013002    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 16:17:24.013028 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.013005    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 16:17:24.013734 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.013716    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 16:17:24.013734 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.013739    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 16:17:24.013888 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.013771    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 16:17:24.014181 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.014163    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xdk5b\""
Sep 11 16:17:24.014181 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.014176    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 16:17:24.035110 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.035055    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rq2t9"
Sep 11 16:17:24.046342 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:24.046324    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rq2t9"
Sep 11 16:17:25.663590 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.662140    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:25.703157 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.703140    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 16:17:25.708162 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708142    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-host\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.708246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-sys-fs\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.708246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708185    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-bin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-etc-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysconfig\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708235    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-kubernetes\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708268    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-systemd\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2pph\" (UniqueName: \"kubernetes.io/projected/495fbc87-c2e0-45b7-b333-4fd655f4c71c-kube-api-access-p2pph\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzss\" (UniqueName: \"kubernetes.io/projected/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-kube-api-access-9nzss\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708322    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-etc-kubernetes\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708346    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708373    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-kubelet-config\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708388    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-env-overrides\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rc6t\" (UniqueName: \"kubernetes.io/projected/a4bbaa63-55aa-4948-8f3c-587253f132af-kube-api-access-2rc6t\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.708468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708441    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-socket-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-system-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708510    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-netns\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-hostroot\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-multus-certs\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708569    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-systemd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-ovn\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708621    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-slash\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708640    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-run\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708655    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-host\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708677    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-tmp\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708698    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708722    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-var-lib-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708736    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-node-log\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708749    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-script-lib\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.708878 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwwf8\" (UniqueName: \"kubernetes.io/projected/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-kube-api-access-dwwf8\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708803    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqbt9\" (UniqueName: \"kubernetes.io/projected/90b33fa7-6adf-4900-91be-9f8c33c12eca-kube-api-access-zqbt9\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5xgz\" (UniqueName: \"kubernetes.io/projected/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kube-api-access-n5xgz\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-daemon-config\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708865    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-modprobe-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a373d3fc-9c59-4ea6-841b-fc50591c663e-iptables-alerter-script\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708916    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-579bq\" (UniqueName: \"kubernetes.io/projected/a373d3fc-9c59-4ea6-841b-fc50591c663e-kube-api-access-579bq\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-device-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-conf-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.708976    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-kubelet\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-system-cni-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709065    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a373d3fc-9c59-4ea6-841b-fc50591c663e-host-slash\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-cni-binary-copy\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.709598 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709110    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709116    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a373d3fc-9c59-4ea6-841b-fc50591c663e-host-slash\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709152    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-lib-modules\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b2k8\" (UniqueName: \"kubernetes.io/projected/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-kube-api-access-4b2k8\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709236    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4bbaa63-55aa-4948-8f3c-587253f132af-hosts-file\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709304    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a4bbaa63-55aa-4948-8f3c-587253f132af-hosts-file\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-netd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709372    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a373d3fc-9c59-4ea6-841b-fc50591c663e-iptables-alerter-script\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709378    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bbaa63-55aa-4948-8f3c-587253f132af-tmp-dir\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709405    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-agent-certs\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709479    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-multus\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709495    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-var-lib-kubelet\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709520    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-tuned\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709536    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovn-node-metrics-cert\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710237 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709563    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-serviceca\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709583    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-registration-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709597    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-cnibin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709603    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a4bbaa63-55aa-4948-8f3c-587253f132af-tmp-dir\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-netns\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709668    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-systemd-units\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709684    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-log-socket\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709698    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-bin\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-konnectivity-ca\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709733    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-k8s-cni-cncf-io\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-kubelet\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709826    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjkfw\" (UniqueName: \"kubernetes.io/projected/7b93e941-5f75-41d9-b3ec-07abf031a20d-kube-api-access-zjkfw\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709849    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-sys\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-config\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709872    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-conf\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cnibin\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.710834 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.711309 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709955    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-etc-selinux\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.711309 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.709970    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-socket-dir-parent\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.711309 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.710004    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-dbus\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.711309 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.710051    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-konnectivity-ca\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:25.713240 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.713220    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e56aab3b-bf82-4e9d-915f-b73e11e5ac5b-agent-certs\") pod \"konnectivity-agent-tcclt\" (UID: \"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b\") " pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:25.715556 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.715537    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:25.715635 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.715560    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:25.715635 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.715570    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:25.715635 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.715627    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.215612175 +0000 UTC m=+1.988852188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:25.717015 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.716995    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-579bq\" (UniqueName: \"kubernetes.io/projected/a373d3fc-9c59-4ea6-841b-fc50591c663e-kube-api-access-579bq\") pod \"iptables-alerter-wsddv\" (UID: \"a373d3fc-9c59-4ea6-841b-fc50591c663e\") " pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.717203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.717186    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rc6t\" (UniqueName: \"kubernetes.io/projected/a4bbaa63-55aa-4948-8f3c-587253f132af-kube-api-access-2rc6t\") pod \"node-resolver-knxgq\" (UID: \"a4bbaa63-55aa-4948-8f3c-587253f132af\") " pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.745405 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.745369    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a631e1211894d68a5cbaa9874d6ed29.slice/crio-a85c33ab83e055faf33b12443bafa1bb4b60af904f76cdeea9cd37d57f5da34e WatchSource:0}: Error finding container a85c33ab83e055faf33b12443bafa1bb4b60af904f76cdeea9cd37d57f5da34e: Status 404 returned error can't find the container with id a85c33ab83e055faf33b12443bafa1bb4b60af904f76cdeea9cd37d57f5da34e
Sep 11 16:17:25.745634 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.745615    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34bdb01b23407b83abb142f3983543d1.slice/crio-4fb6cdd1b1f9e06ba0da068aa1fada99d682e0c91544b4a39b21e69dba5ce670 WatchSource:0}: Error finding container 4fb6cdd1b1f9e06ba0da068aa1fada99d682e0c91544b4a39b21e69dba5ce670: Status 404 returned error can't find the container with id 4fb6cdd1b1f9e06ba0da068aa1fada99d682e0c91544b4a39b21e69dba5ce670
Sep 11 16:17:25.749731 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.749714    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:17:25.810816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810786    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjkfw\" (UniqueName: \"kubernetes.io/projected/7b93e941-5f75-41d9-b3ec-07abf031a20d-kube-api-access-zjkfw\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.810816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810815    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-sys\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-config\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-conf\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810877    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cnibin\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810913    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-sys\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810922    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cnibin\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810950    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-etc-selinux\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.810995    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-socket-dir-parent\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811013    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-dbus\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811034    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-host\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.811041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811033    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-conf\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-sys-fs\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811066    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-socket-dir-parent\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-bin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811111    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-bin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811112    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-etc-selinux\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-etc-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811120    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-dbus\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-sys-fs\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811170    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-etc-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysconfig\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811220    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-kubernetes\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-systemd\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysconfig\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811253    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2pph\" (UniqueName: \"kubernetes.io/projected/495fbc87-c2e0-45b7-b333-4fd655f4c71c-kube-api-access-p2pph\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811289    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzss\" (UniqueName: \"kubernetes.io/projected/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-kube-api-access-9nzss\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811303    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-systemd\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-etc-kubernetes\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.811538 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811322    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-kubernetes\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-kubelet-config\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-etc-kubernetes\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811390    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-env-overrides\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811465    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-socket-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811487    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-config\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-system-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811522    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d2003fb4-89c4-4059-8096-cc104438d87c-kubelet-config\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811553    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-netns\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811563    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811573    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-host\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811585    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-netns\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-hostroot\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-multus-certs\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811642    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.812241 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811649    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-system-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811645    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-systemd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811623    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-hostroot\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811683    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-multus-certs\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811672    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-socket-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-systemd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811716    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-cni-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811740    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-ovn\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-slash\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811785    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-run-ovn\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811793    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-slash\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811797    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-run\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-host\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811837    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-tmp\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811886    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-run\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811907    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-host\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811910    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-var-lib-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-node-log\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811942    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-var-lib-openvswitch\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812195    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-env-overrides\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811993    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-node-log\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.811968    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-script-lib\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwwf8\" (UniqueName: \"kubernetes.io/projected/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-kube-api-access-dwwf8\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqbt9\" (UniqueName: \"kubernetes.io/projected/90b33fa7-6adf-4900-91be-9f8c33c12eca-kube-api-access-zqbt9\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5xgz\" (UniqueName: \"kubernetes.io/projected/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kube-api-access-n5xgz\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812583    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-daemon-config\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812611    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-modprobe-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-modprobe-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.812949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovnkube-script-lib\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813125    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-device-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-conf-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.813915 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-kubelet\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813245    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-kubelet\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813295    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-conf-dir\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813369    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813412    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-system-cni-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813419    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-sysctl-d\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-multus-daemon-config\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813488    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-system-cni-dir\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813537    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-device-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813545    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-cni-binary-copy\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813607    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813623    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813654    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-lib-modules\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b2k8\" (UniqueName: \"kubernetes.io/projected/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-kube-api-access-4b2k8\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.814695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813721    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813748    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-netd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813816    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-multus\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813869    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-var-lib-kubelet\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813901    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-tuned\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovn-node-metrics-cert\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813960    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-serviceca\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.813993    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-registration-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-cnibin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814057    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-netns\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-systemd-units\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-log-socket\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814143    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b93e941-5f75-41d9-b3ec-07abf031a20d-cni-binary-copy\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-bin\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814154    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-binary-copy\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814185    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-k8s-cni-cncf-io\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-registration-dir\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.815997 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.814342    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814354    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-run-k8s-cni-cncf-io\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814360    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-cnibin\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814397    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-kubelet\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814403    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-var-lib-kubelet\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.814478    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.314439803 +0000 UTC m=+2.087679826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814512    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-kubelet\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814630    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-netd\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814767    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-log-socket\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814867    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-systemd-units\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814949    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-run-netns\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814981    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.814310    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-host-cni-bin\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.815036    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b93e941-5f75-41d9-b3ec-07abf031a20d-host-var-lib-cni-multus\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.815136    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.815134    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/495fbc87-c2e0-45b7-b333-4fd655f4c71c-lib-modules\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:25.815198    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:26.315174679 +0000 UTC m=+2.088414704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:25.816678 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.815216    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-serviceca\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.817224 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.815356    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-tmp\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.818002 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.817985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/495fbc87-c2e0-45b7-b333-4fd655f4c71c-etc-tuned\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.818150 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.818133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-ovn-node-metrics-cert\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.826661 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.826635    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjkfw\" (UniqueName: \"kubernetes.io/projected/7b93e941-5f75-41d9-b3ec-07abf031a20d-kube-api-access-zjkfw\") pod \"multus-sqwzq\" (UID: \"7b93e941-5f75-41d9-b3ec-07abf031a20d\") " pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:25.827533 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.827503    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqbt9\" (UniqueName: \"kubernetes.io/projected/90b33fa7-6adf-4900-91be-9f8c33c12eca-kube-api-access-zqbt9\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:25.827627 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.827541    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5xgz\" (UniqueName: \"kubernetes.io/projected/42d1eea9-dcb1-40e5-b9bb-4cb918bd3111-kube-api-access-n5xgz\") pod \"aws-ebs-csi-driver-node-7n6nj\" (UID: \"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:25.827712 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.827692    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwwf8\" (UniqueName: \"kubernetes.io/projected/1aa00ea3-3996-469b-bae6-8c6b7c9fe376-kube-api-access-dwwf8\") pod \"ovnkube-node-9ctcb\" (UID: \"1aa00ea3-3996-469b-bae6-8c6b7c9fe376\") " pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:25.827880 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.827859    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2pph\" (UniqueName: \"kubernetes.io/projected/495fbc87-c2e0-45b7-b333-4fd655f4c71c-kube-api-access-p2pph\") pod \"tuned-k6zb9\" (UID: \"495fbc87-c2e0-45b7-b333-4fd655f4c71c\") " pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.829086 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.829065    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b2k8\" (UniqueName: \"kubernetes.io/projected/3b236a71-d94c-4de5-8fb5-3b75d3c829b3-kube-api-access-4b2k8\") pod \"multus-additional-cni-plugins-68vlf\" (UID: \"3b236a71-d94c-4de5-8fb5-3b75d3c829b3\") " pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:25.829189 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.829172    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzss\" (UniqueName: \"kubernetes.io/projected/d2c8df0b-55b4-4fc4-b451-5e99cfe694ff-kube-api-access-9nzss\") pod \"node-ca-zjzhz\" (UID: \"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff\") " pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.926360 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.926277    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wsddv"
Sep 11 16:17:25.934624 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.934597    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda373d3fc_9c59_4ea6_841b_fc50591c663e.slice/crio-2be44ca2fb3639fa8ee9d85577b90fc846fbfd063bea050b7a2626646705d563 WatchSource:0}: Error finding container 2be44ca2fb3639fa8ee9d85577b90fc846fbfd063bea050b7a2626646705d563: Status 404 returned error can't find the container with id 2be44ca2fb3639fa8ee9d85577b90fc846fbfd063bea050b7a2626646705d563
Sep 11 16:17:25.942960 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.942928    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:25.950491 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.950469    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56aab3b_bf82_4e9d_915f_b73e11e5ac5b.slice/crio-b515cdaecd0400a9a193cde2ed547af2dc5554a28b0afc5a5c64c44a6086ff34 WatchSource:0}: Error finding container b515cdaecd0400a9a193cde2ed547af2dc5554a28b0afc5a5c64c44a6086ff34: Status 404 returned error can't find the container with id b515cdaecd0400a9a193cde2ed547af2dc5554a28b0afc5a5c64c44a6086ff34
Sep 11 16:17:25.956630 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.956612    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-knxgq"
Sep 11 16:17:25.963810 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.963789    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4bbaa63_55aa_4948_8f3c_587253f132af.slice/crio-196387ea54718d6854988a848a558ce1c10ce7307a37211df745e74ca41fcde3 WatchSource:0}: Error finding container 196387ea54718d6854988a848a558ce1c10ce7307a37211df745e74ca41fcde3: Status 404 returned error can't find the container with id 196387ea54718d6854988a848a558ce1c10ce7307a37211df745e74ca41fcde3
Sep 11 16:17:25.966843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.966825    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zjzhz"
Sep 11 16:17:25.973663 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.973508    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c8df0b_55b4_4fc4_b451_5e99cfe694ff.slice/crio-ff7c65e459c2838775cce60006d21f74047539822de37b8613d95582885f51cc WatchSource:0}: Error finding container ff7c65e459c2838775cce60006d21f74047539822de37b8613d95582885f51cc: Status 404 returned error can't find the container with id ff7c65e459c2838775cce60006d21f74047539822de37b8613d95582885f51cc
Sep 11 16:17:25.985214 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:25.985197    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k6zb9"
Sep 11 16:17:25.992581 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:25.992559    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495fbc87_c2e0_45b7_b333_4fd655f4c71c.slice/crio-84c9b053081fd0b93eea759b9a439aa792caf8542db115043d763f0e22313426 WatchSource:0}: Error finding container 84c9b053081fd0b93eea759b9a439aa792caf8542db115043d763f0e22313426: Status 404 returned error can't find the container with id 84c9b053081fd0b93eea759b9a439aa792caf8542db115043d763f0e22313426
Sep 11 16:17:26.000362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.000342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj"
Sep 11 16:17:26.007394 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:26.007373    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d1eea9_dcb1_40e5_b9bb_4cb918bd3111.slice/crio-fcbb3fa3e595bb475e9388bee24d2cd75940e72dc088dced78652a1c7bb86cbe WatchSource:0}: Error finding container fcbb3fa3e595bb475e9388bee24d2cd75940e72dc088dced78652a1c7bb86cbe: Status 404 returned error can't find the container with id fcbb3fa3e595bb475e9388bee24d2cd75940e72dc088dced78652a1c7bb86cbe
Sep 11 16:17:26.028973 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.028941    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sqwzq"
Sep 11 16:17:26.037500 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:26.037476    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b93e941_5f75_41d9_b3ec_07abf031a20d.slice/crio-dfdabf6952ac39ae2da3b95d72646af7d72c125ae7f0a41ca622440d82f58b2d WatchSource:0}: Error finding container dfdabf6952ac39ae2da3b95d72646af7d72c125ae7f0a41ca622440d82f58b2d: Status 404 returned error can't find the container with id dfdabf6952ac39ae2da3b95d72646af7d72c125ae7f0a41ca622440d82f58b2d
Sep 11 16:17:26.039503 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.039488    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-68vlf"
Sep 11 16:17:26.046883 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:26.046862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b236a71_d94c_4de5_8fb5_3b75d3c829b3.slice/crio-71a824ed738b1f6f51c2f1cc9bf18bffae5bf8d74696d9da68f20407a0342558 WatchSource:0}: Error finding container 71a824ed738b1f6f51c2f1cc9bf18bffae5bf8d74696d9da68f20407a0342558: Status 404 returned error can't find the container with id 71a824ed738b1f6f51c2f1cc9bf18bffae5bf8d74696d9da68f20407a0342558
Sep 11 16:17:26.050906 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.050887    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:26.058280 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:26.058259    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aa00ea3_3996_469b_bae6_8c6b7c9fe376.slice/crio-dde80cbef36e74a5d3fdb96cb922009e91cb1b88721df77d5b0b4ed216d61363 WatchSource:0}: Error finding container dde80cbef36e74a5d3fdb96cb922009e91cb1b88721df77d5b0b4ed216d61363: Status 404 returned error can't find the container with id dde80cbef36e74a5d3fdb96cb922009e91cb1b88721df77d5b0b4ed216d61363
Sep 11 16:17:26.132087 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.132057    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:26.217608 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.217441    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:26.217778 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.217617    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:26.217778 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.217637    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:26.217778 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.217650    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:26.217778 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.217705    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.217689087 +0000 UTC m=+2.990929143 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:26.318648 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.318610    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:26.318808 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.318659    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:26.318808 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.318773    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:26.318808 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.318776    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:26.318970 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.318834    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.318814811 +0000 UTC m=+3.092054840 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:26.318970 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.318852    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:27.3188431 +0000 UTC m=+3.092083109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:26.656206 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.656029    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:25 +0000 UTC" deadline="2028-02-15 06:46:09.551635775 +0000 UTC"
Sep 11 16:17:26.656206 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.656094    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12518h28m42.895547349s"
Sep 11 16:17:26.752052 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.746954    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:26.752052 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:26.747070    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:26.771064 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.771007    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal" event={"ID":"9a631e1211894d68a5cbaa9874d6ed29","Type":"ContainerStarted","Data":"a85c33ab83e055faf33b12443bafa1bb4b60af904f76cdeea9cd37d57f5da34e"}
Sep 11 16:17:26.791569 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.791533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"dde80cbef36e74a5d3fdb96cb922009e91cb1b88721df77d5b0b4ed216d61363"}
Sep 11 16:17:26.814976 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.814835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj" event={"ID":"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111","Type":"ContainerStarted","Data":"fcbb3fa3e595bb475e9388bee24d2cd75940e72dc088dced78652a1c7bb86cbe"}
Sep 11 16:17:26.819579 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.819522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6zb9" event={"ID":"495fbc87-c2e0-45b7-b333-4fd655f4c71c","Type":"ContainerStarted","Data":"84c9b053081fd0b93eea759b9a439aa792caf8542db115043d763f0e22313426"}
Sep 11 16:17:26.829835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.829803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjzhz" event={"ID":"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff","Type":"ContainerStarted","Data":"ff7c65e459c2838775cce60006d21f74047539822de37b8613d95582885f51cc"}
Sep 11 16:17:26.843202 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.843162    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-knxgq" event={"ID":"a4bbaa63-55aa-4948-8f3c-587253f132af","Type":"ContainerStarted","Data":"196387ea54718d6854988a848a558ce1c10ce7307a37211df745e74ca41fcde3"}
Sep 11 16:17:26.852989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.852953    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tcclt" event={"ID":"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b","Type":"ContainerStarted","Data":"b515cdaecd0400a9a193cde2ed547af2dc5554a28b0afc5a5c64c44a6086ff34"}
Sep 11 16:17:26.855903 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.855830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wsddv" event={"ID":"a373d3fc-9c59-4ea6-841b-fc50591c663e","Type":"ContainerStarted","Data":"2be44ca2fb3639fa8ee9d85577b90fc846fbfd063bea050b7a2626646705d563"}
Sep 11 16:17:26.858472 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.858399    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal" event={"ID":"34bdb01b23407b83abb142f3983543d1","Type":"ContainerStarted","Data":"4fb6cdd1b1f9e06ba0da068aa1fada99d682e0c91544b4a39b21e69dba5ce670"}
Sep 11 16:17:26.874289 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.874256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerStarted","Data":"71a824ed738b1f6f51c2f1cc9bf18bffae5bf8d74696d9da68f20407a0342558"}
Sep 11 16:17:26.889176 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.889146    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sqwzq" event={"ID":"7b93e941-5f75-41d9-b3ec-07abf031a20d","Type":"ContainerStarted","Data":"dfdabf6952ac39ae2da3b95d72646af7d72c125ae7f0a41ca622440d82f58b2d"}
Sep 11 16:17:26.998517 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:26.996167    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:27.238731 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.238680    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:27.238971 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.238895    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:27.238971 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.238916    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:27.238971 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.238929    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:27.239153 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.238989    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.238969739 +0000 UTC m=+5.012209767 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:27.339518 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.339348    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:27.339518 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.339395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:27.339737 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.339553    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:27.339737 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.339617    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.339599026 +0000 UTC m=+5.112839050 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:27.340094 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.340009    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:27.340094 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.340063    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:29.340048483 +0000 UTC m=+5.113288506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:27.657095 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.656783    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 16:12:25 +0000 UTC" deadline="2028-04-30 23:03:12.192623019 +0000 UTC"
Sep 11 16:17:27.657095 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.656822    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14334h45m44.535805764s"
Sep 11 16:17:27.746210 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.745494    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:27.746210 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.745653    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:27.746210 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:27.746049    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:27.746210 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:27.746157    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:28.182594 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:28.182477    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 16:17:28.746040 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:28.745549    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:28.746040 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:28.745668    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:29.258912 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:29.257111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:29.258912 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.257315    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:29.258912 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.257337    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:29.258912 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.257351    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:29.258912 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.257411    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.257391729 +0000 UTC m=+9.030631757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:29.357988 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:29.357947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:29.357988 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:29.357988    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:29.358228 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.358098    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:29.358228 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.358150    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.358136633 +0000 UTC m=+9.131376655 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:29.359016 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.358520    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:29.359016 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.358565    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:33.358552109 +0000 UTC m=+9.131792131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:29.745681 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:29.744895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:29.745681 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.745051    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:29.745681 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:29.745477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:29.745681 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:29.745580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:30.745680 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:30.745642    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:30.746160 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:30.745788    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:31.745844 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:31.745216    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:31.745844 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:31.745351    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:31.745844 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:31.745417    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:31.745844 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:31.745514    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:32.744860 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:32.744814    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:32.745057 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:32.744964    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:33.294815 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:33.294772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:33.295278 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.295000    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:33.295278 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.295021    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:33.295278 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.295034    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:33.295278 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.295096    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.295078242 +0000 UTC m=+17.068318265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:33.395312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:33.395363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.395508    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.395574    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.395555924 +0000 UTC m=+17.168795932 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.395643    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:33.396063 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.395673    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:41.395665329 +0000 UTC m=+17.168905338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:33.745545 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:33.745509    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:33.745735 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.745641    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:33.746059 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:33.746037    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:33.746163 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:33.746142    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:34.747635 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:34.747586    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:34.748051 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:34.747692    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:35.745733 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:35.745633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:35.745903 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:35.745633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:35.745903 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:35.745785    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:35.745903 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:35.745860    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:36.745563 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:36.745522    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:36.746042 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:36.745657    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:37.745572 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:37.745539    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:37.745998 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:37.745662    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:37.746058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:37.746039    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:37.746146 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:37.746125    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:38.745027 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:38.744986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:38.745227 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:38.745131    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:39.745478 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:39.745423    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:39.745478 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:39.745477    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:39.746062 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:39.745615    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:39.746062 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:39.745732    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:40.745387 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:40.745347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:40.745574 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:40.745506    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:41.361299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:41.361222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:41.361459 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.361405    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:41.361459 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.361429    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:41.361459 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.361439    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:41.361594 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.361513    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.361499036 +0000 UTC m=+33.134739063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:41.462602 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:41.462563    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:41.462602 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:41.462615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:41.462843 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.462724    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:41.462843 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.462724    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:41.462843 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.462794    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.462774215 +0000 UTC m=+33.236014247 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:41.462843 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.462812    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:17:57.46280329 +0000 UTC m=+33.236043305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:41.744942 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:41.744841    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:41.745119 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.744979    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:41.745119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:41.745047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:41.745257 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:41.745166    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:42.745517 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:42.745428    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:42.746411 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:42.745681    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:43.745401 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:43.745356    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:43.745610 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:43.745355    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:43.745610 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:43.745527    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:43.745969 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:43.745609    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:44.745975 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:44.745505    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:44.745975 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:44.745620    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:45.745663 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.745309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:45.745803 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.745520    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:45.745890 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:45.745864    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:45.746213 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:45.746187    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:45.933946 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.933768    2579 generic.go:356] "Generic (PLEG): container finished" podID="9a631e1211894d68a5cbaa9874d6ed29" containerID="11e837e3008d32b5be7c921660ee2fc3dd4c141b1ce22e01c4ffb79f95cf5f7e" exitCode=0
Sep 11 16:17:45.934089 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.933851    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal" event={"ID":"9a631e1211894d68a5cbaa9874d6ed29","Type":"ContainerDied","Data":"11e837e3008d32b5be7c921660ee2fc3dd4c141b1ce22e01c4ffb79f95cf5f7e"}
Sep 11 16:17:45.937060 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.937032    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"276c9b423f1431a3586d55ad2f072f3f6f08c77518107aeaae6418fa837e2b33"}
Sep 11 16:17:45.937141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.937125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"aabd28866fceaa9a97b53acb47d7a18e8260ffa668a2fa2bd110881579322173"}
Sep 11 16:17:45.937191 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.937178    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"dc2115074646da6d8747c750084508a1b1414b7a387a63c74dc3e866781bf06f"}
Sep 11 16:17:45.937238 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.937231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"2126e08691250f5c2e2cb5d03ba5189ff82485150bad14e60a00d5f09071a1e5"}
Sep 11 16:17:45.938564 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.938541    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj" event={"ID":"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111","Type":"ContainerStarted","Data":"43c50d32d61d0532e7b6ae3e3d1ce6e49818efafb0aa5d7769cafee7427a562c"}
Sep 11 16:17:45.940022 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.940001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k6zb9" event={"ID":"495fbc87-c2e0-45b7-b333-4fd655f4c71c","Type":"ContainerStarted","Data":"dc5376c01163f74f318eda13fbb0cde970d255036b39d3a080fe8d3236c8e272"}
Sep 11 16:17:45.944588 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.944556    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zjzhz" event={"ID":"d2c8df0b-55b4-4fc4-b451-5e99cfe694ff","Type":"ContainerStarted","Data":"0cae80f532934b16938ea5d863cde9b6ec098b4781cf9ea43a3a6d4420b14f69"}
Sep 11 16:17:45.946415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.946393    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-knxgq" event={"ID":"a4bbaa63-55aa-4948-8f3c-587253f132af","Type":"ContainerStarted","Data":"6a05de1ef90112919412a87a0a67fdf5cba93791449e426474b35bbb5f275e83"}
Sep 11 16:17:45.947762 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.947742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tcclt" event={"ID":"e56aab3b-bf82-4e9d-915f-b73e11e5ac5b","Type":"ContainerStarted","Data":"b1ae0c903f841b2ff800fc9a552fb6700c73c7bec33ae57171cea606703d657d"}
Sep 11 16:17:45.949085 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.949066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal" event={"ID":"34bdb01b23407b83abb142f3983543d1","Type":"ContainerStarted","Data":"4c6d65d075ce49c88fc498f9a049d062def8b79d3cc1aac3e7bb2d790a8d9681"}
Sep 11 16:17:45.950681 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.950655    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="efb00ea5d49f1ec039b7985068e097f993bf7ba0b737e24557788b58506afafc" exitCode=0
Sep 11 16:17:45.950772 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.950727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"efb00ea5d49f1ec039b7985068e097f993bf7ba0b737e24557788b58506afafc"}
Sep 11 16:17:45.952185 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.952152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sqwzq" event={"ID":"7b93e941-5f75-41d9-b3ec-07abf031a20d","Type":"ContainerStarted","Data":"249694159bda2711adb00b09498760ca156b0b0e45ae33634a7bad4668a5daad"}
Sep 11 16:17:45.991204 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.991151    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zjzhz" podStartSLOduration=2.669091411 podStartE2EDuration="21.991132549s" totalImagesPullingTime="19.322041138s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:45.991102855 +0000 UTC m=+21.764342887" watchObservedRunningTime="2026-09-11 16:17:45.991132549 +0000 UTC m=+21.764372580"
Sep 11 16:17:45.991659 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:45.991587    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-knxgq" podStartSLOduration=2.6366374329999998 podStartE2EDuration="21.991576634s" totalImagesPullingTime="19.354939201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:45.973042957 +0000 UTC m=+21.746282989" watchObservedRunningTime="2026-09-11 16:17:45.991576634 +0000 UTC m=+21.764816665"
Sep 11 16:17:46.012061 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.011999    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k6zb9" podStartSLOduration=2.685153032 podStartE2EDuration="22.011977424s" totalImagesPullingTime="19.326824392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.011661456 +0000 UTC m=+21.784901487" watchObservedRunningTime="2026-09-11 16:17:46.011977424 +0000 UTC m=+21.785217456"
Sep 11 16:17:46.028971 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.028926    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sqwzq" podStartSLOduration=2.645142318 podStartE2EDuration="22.028909967s" totalImagesPullingTime="19.383767649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.0287241 +0000 UTC m=+21.801964132" watchObservedRunningTime="2026-09-11 16:17:46.028909967 +0000 UTC m=+21.802149998"
Sep 11 16:17:46.043019 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.042973    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-18.ec2.internal" podStartSLOduration=21.042957905 podStartE2EDuration="21.042957905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.042763202 +0000 UTC m=+21.816003244" watchObservedRunningTime="2026-09-11 16:17:46.042957905 +0000 UTC m=+21.816197936"
Sep 11 16:17:46.083689 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.083638    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tcclt" podStartSLOduration=2.717246405 podStartE2EDuration="22.083623732s" totalImagesPullingTime="19.366377327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.062420659 +0000 UTC m=+21.835660689" watchObservedRunningTime="2026-09-11 16:17:46.083623732 +0000 UTC m=+21.856863762"
Sep 11 16:17:46.745299 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.745266    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:46.745613 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:46.745484    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:46.960162 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.959682    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal" event={"ID":"9a631e1211894d68a5cbaa9874d6ed29","Type":"ContainerStarted","Data":"9bd770e93b043e9fabe3f22006daa626e2591b133cda64940a77f619c5243a2d"}
Sep 11 16:17:46.968047 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.967971    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"37ee2c1c828dcd83432358684f6fda61a195dc247268a6e65b6421516eb6a452"}
Sep 11 16:17:46.968387 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.968137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"6d1719e0a3b18dac64c84bb01ed5acc4629adc0fda03834ed7bf1d24ebf7b3c3"}
Sep 11 16:17:46.978736 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:46.978649    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-18.ec2.internal" podStartSLOduration=21.978621807 podStartE2EDuration="21.978621807s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:46.977858025 +0000 UTC m=+22.751098051" watchObservedRunningTime="2026-09-11 16:17:46.978621807 +0000 UTC m=+22.751861838"
Sep 11 16:17:47.578648 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.578614    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 16:17:47.648872 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.648831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:47.649495 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.649474    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:47.674626 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.674528    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T16:17:47.578641642Z","UUID":"e39fc2ab-d93d-479c-a8de-a33ba16af109","Handler":null,"Name":"","Endpoint":""}
Sep 11 16:17:47.677333 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.677304    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 16:17:47.677543 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.677349    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 16:17:47.744764 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.744679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:47.745049 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.744680    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:47.745049 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:47.744812    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:47.745049 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:47.744889    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:47.973342 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.973296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wsddv" event={"ID":"a373d3fc-9c59-4ea6-841b-fc50591c663e","Type":"ContainerStarted","Data":"afd00c483f4d093e5fb0e0d0a595ca6d5a8a204bb5a3f3bfc0c58c694fd3ca5f"}
Sep 11 16:17:47.976265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.976054    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj" event={"ID":"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111","Type":"ContainerStarted","Data":"bc2a15ace46db7806ac0bd76e89f0afacce0bd1711c2ec01b6b033ad88d061e4"}
Sep 11 16:17:47.991648 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:47.991591    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wsddv" podStartSLOduration=4.630646624 podStartE2EDuration="23.991573719s" totalImagesPullingTime="19.360927095s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:47.99147843 +0000 UTC m=+23.764718457" watchObservedRunningTime="2026-09-11 16:17:47.991573719 +0000 UTC m=+23.764813749"
Sep 11 16:17:48.745025 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:48.744983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:48.745238 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:48.745132    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:48.983149 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:48.983113    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"dad277a4ae5feea442ff3bed79bcec99e26d76a09e77fcabefc938ccddeb4653"}
Sep 11 16:17:48.983149 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:48.983143    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 16:17:49.745651 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:49.745619    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:49.745831 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:49.745653    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:49.745831 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:49.745751    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:49.745957 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:49.745892    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:50.745324 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.745288    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:50.745745 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:50.745426    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:50.989115 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.988906    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="73ea2066cd0dec1d33472d2cf3d7167dc1fecedf132907a7d4193447762d8970" exitCode=0
Sep 11 16:17:50.989262 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.988998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"73ea2066cd0dec1d33472d2cf3d7167dc1fecedf132907a7d4193447762d8970"}
Sep 11 16:17:50.992904 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.992876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" event={"ID":"1aa00ea3-3996-469b-bae6-8c6b7c9fe376","Type":"ContainerStarted","Data":"d7f6581d70feed5b00cd3a0907005d01b4fe8c171e96b6e20178385d68130421"}
Sep 11 16:17:50.993144 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.993128    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:50.995063 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:50.995037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj" event={"ID":"42d1eea9-dcb1-40e5-b9bb-4cb918bd3111","Type":"ContainerStarted","Data":"511863c95318f490bfa17e6c7b3404d51ef32b94174518337db95f9ec99760f6"}
Sep 11 16:17:51.009955 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.009933    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:51.038413 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.038375    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7n6nj" podStartSLOduration=3.283385944 podStartE2EDuration="27.038362103s" totalImagesPullingTime="23.754976159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:51.03821943 +0000 UTC m=+26.811459468" watchObservedRunningTime="2026-09-11 16:17:51.038362103 +0000 UTC m=+26.811602134"
Sep 11 16:17:51.070479 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.070416    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb" podStartSLOduration=6.795880339 podStartE2EDuration="26.070402137s" totalImagesPullingTime="19.274521798s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:17:51.070099481 +0000 UTC m=+26.843339513" watchObservedRunningTime="2026-09-11 16:17:51.070402137 +0000 UTC m=+26.843642169"
Sep 11 16:17:51.744919 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.744894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:51.744992 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.744926    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:51.745064 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:51.745048    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:51.745195 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:51.745177    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:51.873015 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.872985    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:51.873535 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.873110    2579 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 11 16:17:51.873921 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:51.873902    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tcclt"
Sep 11 16:17:52.000515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.000421    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="a95522ba681d31d3f245549d6b6b63896c5e3769766cf1282687f395ae705c39" exitCode=0
Sep 11 16:17:52.000515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.000480    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"a95522ba681d31d3f245549d6b6b63896c5e3769766cf1282687f395ae705c39"}
Sep 11 16:17:52.001738 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.001707    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:52.001965 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.001771    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:52.017576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.017555    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:17:52.454947 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.454860    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xp82l"]
Sep 11 16:17:52.455119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.455023    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:52.455183 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:52.455132    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:52.455705 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.455679    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r6hs9"]
Sep 11 16:17:52.455820 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.455790    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:52.455956 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:52.455897    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:52.458081 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.458057    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ft8q5"]
Sep 11 16:17:52.458174 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:52.458160    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:52.458277 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:52.458258    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:53.745371 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:53.745342    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:53.746135 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:53.745436    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:54.008113 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:54.007793    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="3cf666f1dd9d6bbb88168713a400a7dcc4348a72ff950eb81b79687edaf9231c" exitCode=0
Sep 11 16:17:54.008254 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:54.007873    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"3cf666f1dd9d6bbb88168713a400a7dcc4348a72ff950eb81b79687edaf9231c"}
Sep 11 16:17:54.746868 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:54.746785    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:54.747321 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:54.746927    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:54.747321 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:54.746984    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:54.747321 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:54.747111    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:55.745576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:55.745543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:55.745756 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:55.745659    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:56.744930 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:56.744887    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:56.745395 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:56.744895    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:56.745395 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:56.745054    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-r6hs9" podUID="90b33fa7-6adf-4900-91be-9f8c33c12eca"
Sep 11 16:17:56.745395 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:56.745091    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ft8q5" podUID="d2003fb4-89c4-4059-8096-cc104438d87c"
Sep 11 16:17:57.393191 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:57.393153    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:57.393409 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.393341    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 16:17:57.393409 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.393364    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 16:17:57.393409 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.393374    2579 projected.go:196] Error preparing data for projected volume kube-api-access-nqq4c for pod openshift-network-diagnostics/network-check-target-xp82l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:57.393611 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.393421    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c podName:6348cc3f-d5db-44a6-91dc-bdc28ce89b6d nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.393407515 +0000 UTC m=+65.166647537 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nqq4c" (UniqueName: "kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c") pod "network-check-target-xp82l" (UID: "6348cc3f-d5db-44a6-91dc-bdc28ce89b6d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 16:17:57.493772 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:57.493725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:57.493772 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:57.493781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:57.494026 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.493859    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:57.494026 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.493883    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:57.494026 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.493941    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret podName:d2003fb4-89c4-4059-8096-cc104438d87c nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.493921405 +0000 UTC m=+65.267161427 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret") pod "global-pull-secret-syncer-ft8q5" (UID: "d2003fb4-89c4-4059-8096-cc104438d87c") : object "kube-system"/"original-pull-secret" not registered
Sep 11 16:17:57.494026 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.493963    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs podName:90b33fa7-6adf-4900-91be-9f8c33c12eca nodeName:}" failed. No retries permitted until 2026-09-11 16:18:29.493951596 +0000 UTC m=+65.267191605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs") pod "network-metrics-daemon-r6hs9" (UID: "90b33fa7-6adf-4900-91be-9f8c33c12eca") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 16:17:57.745478 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:57.745377    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:57.745942 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:57.745532    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xp82l" podUID="6348cc3f-d5db-44a6-91dc-bdc28ce89b6d"
Sep 11 16:17:58.410086 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.410061    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-18.ec2.internal" event="NodeReady"
Sep 11 16:17:58.410261 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.410217    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 16:17:58.475669 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.475632    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4qgsv"]
Sep 11 16:17:58.477725 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.477706    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.480572 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.480546    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-n2g87"]
Sep 11 16:17:58.482425 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.482406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.485196 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.485168    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 16:17:58.485291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.485259    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 16:17:58.485436 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.485418    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gkls5\""
Sep 11 16:17:58.485602 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.485587    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 16:17:58.486166 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.486143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 16:17:58.486279 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.486188    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkwjt\""
Sep 11 16:17:58.486279 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.486143    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 16:17:58.496904 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.496875    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4qgsv"]
Sep 11 16:17:58.500805 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.500775    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2g87"]
Sep 11 16:17:58.602166 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-cert\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.602352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95qx\" (UniqueName: \"kubernetes.io/projected/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-kube-api-access-l95qx\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.602352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ssdh\" (UniqueName: \"kubernetes.io/projected/3d537be6-ce63-4191-9b02-1accc8b6f1d7-kube-api-access-6ssdh\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.602352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602301    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d537be6-ce63-4191-9b02-1accc8b6f1d7-config-volume\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.602352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602330    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d537be6-ce63-4191-9b02-1accc8b6f1d7-tmp-dir\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.602498 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.602359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.702989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.702895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d537be6-ce63-4191-9b02-1accc8b6f1d7-config-volume\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.702989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.702945    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d537be6-ce63-4191-9b02-1accc8b6f1d7-tmp-dir\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.703159 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.703159 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-cert\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.703221 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95qx\" (UniqueName: \"kubernetes.io/projected/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-kube-api-access-l95qx\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.703301 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:58.703228    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:58.703301 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703254    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ssdh\" (UniqueName: \"kubernetes.io/projected/3d537be6-ce63-4191-9b02-1accc8b6f1d7-kube-api-access-6ssdh\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.703301 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:58.703296    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls podName:3d537be6-ce63-4191-9b02-1accc8b6f1d7 nodeName:}" failed. No retries permitted until 2026-09-11 16:17:59.203276553 +0000 UTC m=+34.976516565 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls") pod "dns-default-n2g87" (UID: "3d537be6-ce63-4191-9b02-1accc8b6f1d7") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:58.703425 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703232    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d537be6-ce63-4191-9b02-1accc8b6f1d7-tmp-dir\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.703489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.703477    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d537be6-ce63-4191-9b02-1accc8b6f1d7-config-volume\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.707671 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.707647    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-cert\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.713946 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.713917    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95qx\" (UniqueName: \"kubernetes.io/projected/1cbdac5e-efe7-4a77-981a-ea193e51a9fb-kube-api-access-l95qx\") pod \"ingress-canary-4qgsv\" (UID: \"1cbdac5e-efe7-4a77-981a-ea193e51a9fb\") " pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:58.714465 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.714426    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ssdh\" (UniqueName: \"kubernetes.io/projected/3d537be6-ce63-4191-9b02-1accc8b6f1d7-kube-api-access-6ssdh\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:58.745576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.745544    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:17:58.745953 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.745543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:17:58.748672 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.748647    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:17:58.748832 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.748716    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:17:58.749244 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.749223    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\""
Sep 11 16:17:58.788325 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:58.788283    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4qgsv"
Sep 11 16:17:59.024323 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.024074    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="51fb772f0f5b5b352092a5776bb1d1c7514f12562a3f948acda5825949d89d6d" exitCode=0
Sep 11 16:17:59.024323 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.024154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"51fb772f0f5b5b352092a5776bb1d1c7514f12562a3f948acda5825949d89d6d"}
Sep 11 16:17:59.071624 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.071588    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4qgsv"]
Sep 11 16:17:59.076346 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:17:59.076318    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cbdac5e_efe7_4a77_981a_ea193e51a9fb.slice/crio-8dcf450bbd92ee2ceb09d094ab06a57b1034151701564e2818bab39c87d48cda WatchSource:0}: Error finding container 8dcf450bbd92ee2ceb09d094ab06a57b1034151701564e2818bab39c87d48cda: Status 404 returned error can't find the container with id 8dcf450bbd92ee2ceb09d094ab06a57b1034151701564e2818bab39c87d48cda
Sep 11 16:17:59.207932 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.207893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:17:59.208087 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:59.208033    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:17:59.208127 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:17:59.208093    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls podName:3d537be6-ce63-4191-9b02-1accc8b6f1d7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:00.20807834 +0000 UTC m=+35.981318373 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls") pod "dns-default-n2g87" (UID: "3d537be6-ce63-4191-9b02-1accc8b6f1d7") : secret "dns-default-metrics-tls" not found
Sep 11 16:17:59.744714 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.744677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:17:59.747619 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.747589    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\""
Sep 11 16:17:59.748172 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.747601    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:17:59.748622 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:17:59.748604    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:18:00.030948 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:00.030865    2579 generic.go:356] "Generic (PLEG): container finished" podID="3b236a71-d94c-4de5-8fb5-3b75d3c829b3" containerID="5dadc0735c4c0201c1f3a4ef8bec8ba6c23fde27b81e52fab6fe9b5bb8008efe" exitCode=0
Sep 11 16:18:00.030948 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:00.030927    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerDied","Data":"5dadc0735c4c0201c1f3a4ef8bec8ba6c23fde27b81e52fab6fe9b5bb8008efe"}
Sep 11 16:18:00.032560 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:00.032498    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4qgsv" event={"ID":"1cbdac5e-efe7-4a77-981a-ea193e51a9fb","Type":"ContainerStarted","Data":"8dcf450bbd92ee2ceb09d094ab06a57b1034151701564e2818bab39c87d48cda"}
Sep 11 16:18:00.217550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:00.217499    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:00.217734 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:00.217673    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:18:00.217813 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:00.217748    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls podName:3d537be6-ce63-4191-9b02-1accc8b6f1d7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.217731872 +0000 UTC m=+37.990971885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls") pod "dns-default-n2g87" (UID: "3d537be6-ce63-4191-9b02-1accc8b6f1d7") : secret "dns-default-metrics-tls" not found
Sep 11 16:18:01.037912 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.037874    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-68vlf" event={"ID":"3b236a71-d94c-4de5-8fb5-3b75d3c829b3","Type":"ContainerStarted","Data":"6759a66165cbcc28eddbe6d2b774a40c3ccc009ce0a2448eafd90f4ff34afdd3"}
Sep 11 16:18:01.063121 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.062908    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-68vlf" podStartSLOduration=7.574351135 podStartE2EDuration="37.062884594s" totalImagesPullingTime="29.488533459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:01.061738544 +0000 UTC m=+36.834978575" watchObservedRunningTime="2026-09-11 16:18:01.062884594 +0000 UTC m=+36.836124626"
Sep 11 16:18:01.435006 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.434974    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m82tz"]
Sep 11 16:18:01.453488 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.453437    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m82tz"]
Sep 11 16:18:01.453668 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.453559    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.456906 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.456878    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ddmk9\""
Sep 11 16:18:01.457157 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.457141    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 16:18:01.458335 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.458316    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 16:18:01.458979 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.458962    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 16:18:01.460995 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.460979    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 16:18:01.526006 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.525966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c3f8709-56c0-4c3a-887b-2b74dfe217de-crio-socket\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.526006 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.526011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.526203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.526074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c3f8709-56c0-4c3a-887b-2b74dfe217de-data-volume\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.526203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.526089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.526203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.526124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86wvn\" (UniqueName: \"kubernetes.io/projected/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-api-access-86wvn\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.626764 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.626723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c3f8709-56c0-4c3a-887b-2b74dfe217de-crio-socket\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.626764 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.626770    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.626989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.626830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c3f8709-56c0-4c3a-887b-2b74dfe217de-data-volume\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.626989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.626852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.626989 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.626900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86wvn\" (UniqueName: \"kubernetes.io/projected/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-api-access-86wvn\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.627114 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:01.626937    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:01.627163 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.627022    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2c3f8709-56c0-4c3a-887b-2b74dfe217de-crio-socket\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.627163 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.627152    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c3f8709-56c0-4c3a-887b-2b74dfe217de-data-volume\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:01.627245 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:01.627018    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:01.627245 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:01.627169    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.127151902 +0000 UTC m=+37.900391940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:01.627245 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:01.627235    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:02.127217403 +0000 UTC m=+37.900457418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:01.636277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:01.636254    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86wvn\" (UniqueName: \"kubernetes.io/projected/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-api-access-86wvn\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:02.042316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.042281    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4qgsv" event={"ID":"1cbdac5e-efe7-4a77-981a-ea193e51a9fb","Type":"ContainerStarted","Data":"c76bc3661d1b38b48bc7cd06c6f0b2ca38aa080fa65e74bfc5c1138128d0e7f6"}
Sep 11 16:18:02.068119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.068063    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4qgsv" podStartSLOduration=2.025118016 podStartE2EDuration="4.068044835s" totalImagesPullingTime="2.042926819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:02.066869547 +0000 UTC m=+37.840109590" watchObservedRunningTime="2026-09-11 16:18:02.068044835 +0000 UTC m=+37.841284866"
Sep 11 16:18:02.130288 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.130251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:02.130542 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.130329    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:02.130542 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.130429    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:02.130542 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.130522    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:02.130703 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.130587    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.130567457 +0000 UTC m=+38.903807468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:02.130703 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.130607    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:03.130598852 +0000 UTC m=+38.903838864 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:02.231204 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.231168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:02.231335 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.231315    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:18:02.231395 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:02.231384    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls podName:3d537be6-ce63-4191-9b02-1accc8b6f1d7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:06.231367258 +0000 UTC m=+42.004607270 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls") pod "dns-default-n2g87" (UID: "3d537be6-ce63-4191-9b02-1accc8b6f1d7") : secret "dns-default-metrics-tls" not found
Sep 11 16:18:02.345081 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.345006    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"]
Sep 11 16:18:02.379225 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.379182    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"]
Sep 11 16:18:02.379389 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.379290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"
Sep 11 16:18:02.382031 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.382010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 16:18:02.382138 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.382010    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 16:18:02.382138 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.382010    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-rp6st\""
Sep 11 16:18:02.432603 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.432560    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjdh\" (UniqueName: \"kubernetes.io/projected/2f06dc12-13cb-4ebb-8a4e-9e894cada77b-kube-api-access-qzjdh\") pod \"migrator-8b4d697c5-7k47d\" (UID: \"2f06dc12-13cb-4ebb-8a4e-9e894cada77b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"
Sep 11 16:18:02.533030 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.532980    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjdh\" (UniqueName: \"kubernetes.io/projected/2f06dc12-13cb-4ebb-8a4e-9e894cada77b-kube-api-access-qzjdh\") pod \"migrator-8b4d697c5-7k47d\" (UID: \"2f06dc12-13cb-4ebb-8a4e-9e894cada77b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"
Sep 11 16:18:02.542094 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.542067    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjdh\" (UniqueName: \"kubernetes.io/projected/2f06dc12-13cb-4ebb-8a4e-9e894cada77b-kube-api-access-qzjdh\") pod \"migrator-8b4d697c5-7k47d\" (UID: \"2f06dc12-13cb-4ebb-8a4e-9e894cada77b\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"
Sep 11 16:18:02.688611 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.688563    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"
Sep 11 16:18:02.967925 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:02.967817    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d"]
Sep 11 16:18:02.971659 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:02.971624    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f06dc12_13cb_4ebb_8a4e_9e894cada77b.slice/crio-d81085032365a7299d6b8c7a46ac1ab8441b1e95c6728dee3c6213c70d3aa681 WatchSource:0}: Error finding container d81085032365a7299d6b8c7a46ac1ab8441b1e95c6728dee3c6213c70d3aa681: Status 404 returned error can't find the container with id d81085032365a7299d6b8c7a46ac1ab8441b1e95c6728dee3c6213c70d3aa681
Sep 11 16:18:03.045690 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:03.045653    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d" event={"ID":"2f06dc12-13cb-4ebb-8a4e-9e894cada77b","Type":"ContainerStarted","Data":"d81085032365a7299d6b8c7a46ac1ab8441b1e95c6728dee3c6213c70d3aa681"}
Sep 11 16:18:03.138596 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:03.138554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:03.138789 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:03.138619    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:03.138789 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:03.138683    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:03.138789 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:03.138754    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:03.138789 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:03.138764    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.138743592 +0000 UTC m=+40.911983619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:03.138992 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:03.138797    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:05.138784132 +0000 UTC m=+40.912024170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:03.354544 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:03.354415    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-knxgq_a4bbaa63-55aa-4948-8f3c-587253f132af/dns-node-resolver/0.log"
Sep 11 16:18:04.126004 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.125969    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-87vs9"]
Sep 11 16:18:04.144734 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.144694    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-87vs9"]
Sep 11 16:18:04.144897 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.144709    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.147889 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.147858    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 16:18:04.148100 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.148084    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 16:18:04.148442 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.148422    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 16:18:04.148585 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.148563    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 16:18:04.149036 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.149017    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ggpwx\""
Sep 11 16:18:04.248169 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.248134    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-key\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.248343 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.248239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-cabundle\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.248343 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.248262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pb9t\" (UniqueName: \"kubernetes.io/projected/cb4eb0c4-6e05-48ee-bce5-427528c93977-kube-api-access-7pb9t\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.348720 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.348684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-key\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.348884 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.348801    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-cabundle\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.348884 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.348830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pb9t\" (UniqueName: \"kubernetes.io/projected/cb4eb0c4-6e05-48ee-bce5-427528c93977-kube-api-access-7pb9t\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.349416 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.349396    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-cabundle\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.352244 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.352223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/cb4eb0c4-6e05-48ee-bce5-427528c93977-signing-key\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.357962 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.357937    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pb9t\" (UniqueName: \"kubernetes.io/projected/cb4eb0c4-6e05-48ee-bce5-427528c93977-kube-api-access-7pb9t\") pod \"service-ca-568869fdc5-87vs9\" (UID: \"cb4eb0c4-6e05-48ee-bce5-427528c93977\") " pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.457180 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.457094    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-87vs9"
Sep 11 16:18:04.554732 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.554705    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zjzhz_d2c8df0b-55b4-4fc4-b451-5e99cfe694ff/node-ca/0.log"
Sep 11 16:18:04.836581 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.836480    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-87vs9"]
Sep 11 16:18:04.839937 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:04.839909    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4eb0c4_6e05_48ee_bce5_427528c93977.slice/crio-67220918c1aa938dc40b0f29e66a6adde6baa14060e3416efa531b26d6ad5bd6 WatchSource:0}: Error finding container 67220918c1aa938dc40b0f29e66a6adde6baa14060e3416efa531b26d6ad5bd6: Status 404 returned error can't find the container with id 67220918c1aa938dc40b0f29e66a6adde6baa14060e3416efa531b26d6ad5bd6
Sep 11 16:18:04.954547 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:04.954507    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4qgsv_1cbdac5e-efe7-4a77-981a-ea193e51a9fb/serve-healthcheck-canary/0.log"
Sep 11 16:18:05.055095 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.055004    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d" event={"ID":"2f06dc12-13cb-4ebb-8a4e-9e894cada77b","Type":"ContainerStarted","Data":"7a3e218636db850a2fc5f4d57d23070e48c31029c279e99462505165f14b2d28"}
Sep 11 16:18:05.055095 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.055085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d" event={"ID":"2f06dc12-13cb-4ebb-8a4e-9e894cada77b","Type":"ContainerStarted","Data":"e3b61d8d67e1f809f841bcc400d3a45cc9124a9860780b8bb5c03fea7e12df00"}
Sep 11 16:18:05.056169 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.056143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-87vs9" event={"ID":"cb4eb0c4-6e05-48ee-bce5-427528c93977","Type":"ContainerStarted","Data":"67220918c1aa938dc40b0f29e66a6adde6baa14060e3416efa531b26d6ad5bd6"}
Sep 11 16:18:05.076528 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.076481    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-7k47d" podStartSLOduration=1.4880582150000001 podStartE2EDuration="3.076467204s" totalImagesPullingTime="1.588408989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:05.075880071 +0000 UTC m=+40.849120103" watchObservedRunningTime="2026-09-11 16:18:05.076467204 +0000 UTC m=+40.849707226"
Sep 11 16:18:05.155566 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.155533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:05.155879 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:05.155628    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:05.155879 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:05.155676    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:05.155879 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:05.155713    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 16:18:05.155879 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:05.155743    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.155727834 +0000 UTC m=+44.928967881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:05.155879 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:05.155761    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:09.155750212 +0000 UTC m=+44.928990239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : configmap "kube-rbac-proxy" not found
Sep 11 16:18:06.266592 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:06.266548    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:06.267019 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:06.266684    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 16:18:06.267019 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:06.266751    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls podName:3d537be6-ce63-4191-9b02-1accc8b6f1d7 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:14.266733396 +0000 UTC m=+50.039973417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls") pod "dns-default-n2g87" (UID: "3d537be6-ce63-4191-9b02-1accc8b6f1d7") : secret "dns-default-metrics-tls" not found
Sep 11 16:18:08.066470 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:08.066412    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-87vs9" event={"ID":"cb4eb0c4-6e05-48ee-bce5-427528c93977","Type":"ContainerStarted","Data":"595ac4dca7a0d55b16a23bcaa13ae5a89d6a5c9fdf2ed1188f49c39588290890"}
Sep 11 16:18:08.083650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:08.083596    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-87vs9" podStartSLOduration=1.835849735 podStartE2EDuration="4.08357722s" totalImagesPullingTime="2.247727485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:08.08282679 +0000 UTC m=+43.856066823" watchObservedRunningTime="2026-09-11 16:18:08.08357722 +0000 UTC m=+43.856817254"
Sep 11 16:18:09.194395 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:09.194328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:09.194846 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:09.194437    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:09.194846 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:09.194579    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:09.194846 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:09.194649    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls podName:2c3f8709-56c0-4c3a-887b-2b74dfe217de nodeName:}" failed. No retries permitted until 2026-09-11 16:18:17.194631464 +0000 UTC m=+52.967871477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m82tz" (UID: "2c3f8709-56c0-4c3a-887b-2b74dfe217de") : secret "insights-runtime-extractor-tls" not found
Sep 11 16:18:09.195208 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:09.195185    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2c3f8709-56c0-4c3a-887b-2b74dfe217de-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:14.337248 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:14.337210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:14.339654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:14.339628    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d537be6-ce63-4191-9b02-1accc8b6f1d7-metrics-tls\") pod \"dns-default-n2g87\" (UID: \"3d537be6-ce63-4191-9b02-1accc8b6f1d7\") " pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:14.392994 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:14.392962    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:14.694748 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:14.694719    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-n2g87"]
Sep 11 16:18:14.699210 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:14.699185    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d537be6_ce63_4191_9b02_1accc8b6f1d7.slice/crio-075350e031719a63aa4578f85afb1848ebe302c824170a0956cbb620e969e2e2 WatchSource:0}: Error finding container 075350e031719a63aa4578f85afb1848ebe302c824170a0956cbb620e969e2e2: Status 404 returned error can't find the container with id 075350e031719a63aa4578f85afb1848ebe302c824170a0956cbb620e969e2e2
Sep 11 16:18:15.085435 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:15.085357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2g87" event={"ID":"3d537be6-ce63-4191-9b02-1accc8b6f1d7","Type":"ContainerStarted","Data":"075350e031719a63aa4578f85afb1848ebe302c824170a0956cbb620e969e2e2"}
Sep 11 16:18:17.095428 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.095392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2g87" event={"ID":"3d537be6-ce63-4191-9b02-1accc8b6f1d7","Type":"ContainerStarted","Data":"c13bb83058b374f18a26d7103ac1df150504c74ecdafbe89660b5276b500f0aa"}
Sep 11 16:18:17.095990 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.095441    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-n2g87" event={"ID":"3d537be6-ce63-4191-9b02-1accc8b6f1d7","Type":"ContainerStarted","Data":"111efa4dabb516e33beb6040b4e044158476487dfd1e2a2051fcec076f528e03"}
Sep 11 16:18:17.095990 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.095533    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:17.258892 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.258852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:17.261312 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.261283    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2c3f8709-56c0-4c3a-887b-2b74dfe217de-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m82tz\" (UID: \"2c3f8709-56c0-4c3a-887b-2b74dfe217de\") " pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:17.363714 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.363687    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m82tz"
Sep 11 16:18:17.636736 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.636629    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-n2g87" podStartSLOduration=18.026783127 podStartE2EDuration="19.636605245s" totalImagesPullingTime="1.609822118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:17.122096942 +0000 UTC m=+52.895336973" watchObservedRunningTime="2026-09-11 16:18:17.636605245 +0000 UTC m=+53.409845277"
Sep 11 16:18:17.636935 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:17.636918    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m82tz"]
Sep 11 16:18:17.639918 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:17.639893    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3f8709_56c0_4c3a_887b_2b74dfe217de.slice/crio-cfe6ccda4ad17ced5f9889282bbdc6bd5eb43091c1720b086d9adb3fabb61a65 WatchSource:0}: Error finding container cfe6ccda4ad17ced5f9889282bbdc6bd5eb43091c1720b086d9adb3fabb61a65: Status 404 returned error can't find the container with id cfe6ccda4ad17ced5f9889282bbdc6bd5eb43091c1720b086d9adb3fabb61a65
Sep 11 16:18:18.100459 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:18.100353    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m82tz" event={"ID":"2c3f8709-56c0-4c3a-887b-2b74dfe217de","Type":"ContainerStarted","Data":"8bc747f257735e9fc82c802048d9b9012707814043c18228425c4538ff3c2bf5"}
Sep 11 16:18:18.100459 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:18.100406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m82tz" event={"ID":"2c3f8709-56c0-4c3a-887b-2b74dfe217de","Type":"ContainerStarted","Data":"cfe6ccda4ad17ced5f9889282bbdc6bd5eb43091c1720b086d9adb3fabb61a65"}
Sep 11 16:18:19.105993 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:19.105956    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m82tz" event={"ID":"2c3f8709-56c0-4c3a-887b-2b74dfe217de","Type":"ContainerStarted","Data":"3410de2b104da8534982e01b85bc3e294a07582150bd57976a087dee17252047"}
Sep 11 16:18:20.113797 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:20.113764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m82tz" event={"ID":"2c3f8709-56c0-4c3a-887b-2b74dfe217de","Type":"ContainerStarted","Data":"5d86775fc6f64d4e3536d28d9cb952200e668c5824b8d2dce08b19b7fa4c153a"}
Sep 11 16:18:20.133182 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:20.133139    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m82tz" podStartSLOduration=17.198416421 podStartE2EDuration="19.133123447s" totalImagesPullingTime="1.934707026s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:20.132108438 +0000 UTC m=+55.905348469" watchObservedRunningTime="2026-09-11 16:18:20.133123447 +0000 UTC m=+55.906363469"
Sep 11 16:18:24.019111 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:24.019082    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9ctcb"
Sep 11 16:18:26.219463 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.219399    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"]
Sep 11 16:18:26.225506 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.225413    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.229986 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.229934    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 16:18:26.230509 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.230416    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 16:18:26.230697 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.230568    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 16:18:26.230697 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.230601    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 16:18:26.230697 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.230626    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 16:18:26.231167 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.231134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 16:18:26.231167 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.231165    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 16:18:26.245493 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.245390    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"]
Sep 11 16:18:26.324251 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324213    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d9f090-ee36-48d0-b04f-49dbc6638e06-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.324473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v92t\" (UniqueName: \"kubernetes.io/projected/c9d9f090-ee36-48d0-b04f-49dbc6638e06-kube-api-access-4v92t\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.324473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324312    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.324473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.324473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324349    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.324473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.324371    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.330947 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.330916    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cbcdf84f-g8gp4"]
Sep 11 16:18:26.332811 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.332797    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.342401 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.342382    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 16:18:26.342706 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.342693    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 16:18:26.343763 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.343749    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 16:18:26.347786 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.347771    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jml27\""
Sep 11 16:18:26.352940 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.352919    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 16:18:26.365654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.365618    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbcdf84f-g8gp4"]
Sep 11 16:18:26.425105 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v92t\" (UniqueName: \"kubernetes.io/projected/c9d9f090-ee36-48d0-b04f-49dbc6638e06-kube-api-access-4v92t\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-tls\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9dcd92b-6438-41d1-8180-d84ec5fdf475-ca-trust-extracted\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425243    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-bound-sa-token\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425308 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425283    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p9ff\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-kube-api-access-6p9ff\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425395    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-trusted-ca\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425427    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-certificates\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-image-registry-private-configuration\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-installation-pull-secrets\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.425715 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.425581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d9f090-ee36-48d0-b04f-49dbc6638e06-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.426208 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.426171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9d9f090-ee36-48d0-b04f-49dbc6638e06-additional-service-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.427999 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.427940    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.428122 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.428029    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.428215 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.428197    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-ca\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.428408 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.428391    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9d9f090-ee36-48d0-b04f-49dbc6638e06-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.437132 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.437108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v92t\" (UniqueName: \"kubernetes.io/projected/c9d9f090-ee36-48d0-b04f-49dbc6638e06-kube-api-access-4v92t\") pod \"cluster-proxy-proxy-agent-84665bcb95-l5glt\" (UID: \"c9d9f090-ee36-48d0-b04f-49dbc6638e06\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.526843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.526752    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p9ff\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-kube-api-access-6p9ff\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.526843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.526791    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-trusted-ca\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.526843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.526813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-certificates\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.526843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.526841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-image-registry-private-configuration\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.527152 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.526864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-installation-pull-secrets\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.527364 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.527318    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-tls\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.527497 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.527479    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9dcd92b-6438-41d1-8180-d84ec5fdf475-ca-trust-extracted\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.527573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.527525    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-bound-sa-token\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.528260 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.528225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-certificates\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.528663 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.528641    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d9dcd92b-6438-41d1-8180-d84ec5fdf475-trusted-ca\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.530907 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.530802    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d9dcd92b-6438-41d1-8180-d84ec5fdf475-ca-trust-extracted\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.533156 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.533133    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-registry-tls\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.533589 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.533569    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-image-registry-private-configuration\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.533676 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.533621    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d9dcd92b-6438-41d1-8180-d84ec5fdf475-installation-pull-secrets\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.543749 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.543725    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-bound-sa-token\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.543858 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.543779    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p9ff\" (UniqueName: \"kubernetes.io/projected/d9dcd92b-6438-41d1-8180-d84ec5fdf475-kube-api-access-6p9ff\") pod \"image-registry-6cbcdf84f-g8gp4\" (UID: \"d9dcd92b-6438-41d1-8180-d84ec5fdf475\") " pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.557580 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.557548    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"
Sep 11 16:18:26.642340 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.642296    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:26.849681 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.849604    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt"]
Sep 11 16:18:26.853055 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:26.853032    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9d9f090_ee36_48d0_b04f_49dbc6638e06.slice/crio-7e30463349088ffeab12276c7a9d022610febafcf45beddfe6c0ad5c99146703 WatchSource:0}: Error finding container 7e30463349088ffeab12276c7a9d022610febafcf45beddfe6c0ad5c99146703: Status 404 returned error can't find the container with id 7e30463349088ffeab12276c7a9d022610febafcf45beddfe6c0ad5c99146703
Sep 11 16:18:26.916641 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:26.916611    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cbcdf84f-g8gp4"]
Sep 11 16:18:26.919419 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:26.919394    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9dcd92b_6438_41d1_8180_d84ec5fdf475.slice/crio-1d555cb8cf072da398d0a2286f3f022271fc505fadbd7678167ff972ea63cfac WatchSource:0}: Error finding container 1d555cb8cf072da398d0a2286f3f022271fc505fadbd7678167ff972ea63cfac: Status 404 returned error can't find the container with id 1d555cb8cf072da398d0a2286f3f022271fc505fadbd7678167ff972ea63cfac
Sep 11 16:18:27.102675 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.102586    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-n2g87"
Sep 11 16:18:27.142036 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.139427    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt" event={"ID":"c9d9f090-ee36-48d0-b04f-49dbc6638e06","Type":"ContainerStarted","Data":"7e30463349088ffeab12276c7a9d022610febafcf45beddfe6c0ad5c99146703"}
Sep 11 16:18:27.142036 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.141483    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4" event={"ID":"d9dcd92b-6438-41d1-8180-d84ec5fdf475","Type":"ContainerStarted","Data":"7b1c6e9659fbb48f1bf9dc5491ec57655bde3dfcadb2e7fe921574307dd9a1de"}
Sep 11 16:18:27.142036 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.141522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4" event={"ID":"d9dcd92b-6438-41d1-8180-d84ec5fdf475","Type":"ContainerStarted","Data":"1d555cb8cf072da398d0a2286f3f022271fc505fadbd7678167ff972ea63cfac"}
Sep 11 16:18:27.142036 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.141598    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:27.163809 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:27.163750    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4" podStartSLOduration=1.163731708 podStartE2EDuration="1.163731708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:27.163004622 +0000 UTC m=+62.936244655" watchObservedRunningTime="2026-09-11 16:18:27.163731708 +0000 UTC m=+62.936971739"
Sep 11 16:18:29.453620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.453574    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:18:29.457144 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.457116    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 16:18:29.467994 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.467965    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 16:18:29.478046 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.478014    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqq4c\" (UniqueName: \"kubernetes.io/projected/6348cc3f-d5db-44a6-91dc-bdc28ce89b6d-kube-api-access-nqq4c\") pod \"network-check-target-xp82l\" (UID: \"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d\") " pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:18:29.554750 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.554663    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:18:29.554750 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.554760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:18:29.558800 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.558737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 16:18:29.559370 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.559347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 16:18:29.568818 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.568781    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90b33fa7-6adf-4900-91be-9f8c33c12eca-metrics-certs\") pod \"network-metrics-daemon-r6hs9\" (UID: \"90b33fa7-6adf-4900-91be-9f8c33c12eca\") " pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:18:29.569518 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.569494    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d2003fb4-89c4-4059-8096-cc104438d87c-original-pull-secret\") pod \"global-pull-secret-syncer-ft8q5\" (UID: \"d2003fb4-89c4-4059-8096-cc104438d87c\") " pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:18:29.655862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.655820    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ft8q5"
Sep 11 16:18:29.664236 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.664211    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7kds9\""
Sep 11 16:18:29.671046 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.671003    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-r6hs9"
Sep 11 16:18:29.757351 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.757272    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-lgws6\""
Sep 11 16:18:29.765439 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:29.765409    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:18:30.396566 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:30.396477    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-r6hs9"]
Sep 11 16:18:30.406645 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:30.406615    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xp82l"]
Sep 11 16:18:30.415643 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:30.415615    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ft8q5"]
Sep 11 16:18:30.516573 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:30.516539    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90b33fa7_6adf_4900_91be_9f8c33c12eca.slice/crio-b6aaa02ef06c06d0b5c3755055280481827fc9252c998b497b03040512d091de WatchSource:0}: Error finding container b6aaa02ef06c06d0b5c3755055280481827fc9252c998b497b03040512d091de: Status 404 returned error can't find the container with id b6aaa02ef06c06d0b5c3755055280481827fc9252c998b497b03040512d091de
Sep 11 16:18:30.517330 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:30.517302    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6348cc3f_d5db_44a6_91dc_bdc28ce89b6d.slice/crio-4555da946666856a03c05d505a4747c0671c8e040f1b890fb91c1381fdb01b3c WatchSource:0}: Error finding container 4555da946666856a03c05d505a4747c0671c8e040f1b890fb91c1381fdb01b3c: Status 404 returned error can't find the container with id 4555da946666856a03c05d505a4747c0671c8e040f1b890fb91c1381fdb01b3c
Sep 11 16:18:30.518290 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:30.518206    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2003fb4_89c4_4059_8096_cc104438d87c.slice/crio-9ec20b6c58329022b640154908b0aedaf228b4f1973684ce8c549ffaed10289c WatchSource:0}: Error finding container 9ec20b6c58329022b640154908b0aedaf228b4f1973684ce8c549ffaed10289c: Status 404 returned error can't find the container with id 9ec20b6c58329022b640154908b0aedaf228b4f1973684ce8c549ffaed10289c
Sep 11 16:18:31.163782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.163736    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r6hs9" event={"ID":"90b33fa7-6adf-4900-91be-9f8c33c12eca","Type":"ContainerStarted","Data":"b6aaa02ef06c06d0b5c3755055280481827fc9252c998b497b03040512d091de"}
Sep 11 16:18:31.169123 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.169028    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt" event={"ID":"c9d9f090-ee36-48d0-b04f-49dbc6638e06","Type":"ContainerStarted","Data":"a553830b89ad109bbadd87b9d57690da87560973db48f349ce2471a049dbdee2"}
Sep 11 16:18:31.169123 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.169076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt" event={"ID":"c9d9f090-ee36-48d0-b04f-49dbc6638e06","Type":"ContainerStarted","Data":"55ec9f142764bae0e107511bf0b2ed784f41d95f4ff555ba0b96516d09cb058e"}
Sep 11 16:18:31.169123 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.169090    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt" event={"ID":"c9d9f090-ee36-48d0-b04f-49dbc6638e06","Type":"ContainerStarted","Data":"ca73f7d4a5cd132163a2ecbaa417d7e4d265a6125a578ec72898c31ebc6a2ef2"}
Sep 11 16:18:31.172936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.172900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ft8q5" event={"ID":"d2003fb4-89c4-4059-8096-cc104438d87c","Type":"ContainerStarted","Data":"9ec20b6c58329022b640154908b0aedaf228b4f1973684ce8c549ffaed10289c"}
Sep 11 16:18:31.174897 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.174863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xp82l" event={"ID":"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d","Type":"ContainerStarted","Data":"4555da946666856a03c05d505a4747c0671c8e040f1b890fb91c1381fdb01b3c"}
Sep 11 16:18:31.193140 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:31.193078    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-84665bcb95-l5glt" podStartSLOduration=1.483397201 podStartE2EDuration="5.193055599s" totalImagesPullingTime="3.709658398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:31.192429775 +0000 UTC m=+66.965669817" watchObservedRunningTime="2026-09-11 16:18:31.193055599 +0000 UTC m=+66.966295631"
Sep 11 16:18:32.184744 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:32.184654    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r6hs9" event={"ID":"90b33fa7-6adf-4900-91be-9f8c33c12eca","Type":"ContainerStarted","Data":"32e61a78f5dec2285632a1129a0e773c54359df805ff311b065d5f453a6023d8"}
Sep 11 16:18:33.191575 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:33.191522    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-r6hs9" event={"ID":"90b33fa7-6adf-4900-91be-9f8c33c12eca","Type":"ContainerStarted","Data":"77e69e27bcdffb387dc38879acb935bb27b45ba205969a97ab01c22486ef43d4"}
Sep 11 16:18:33.212197 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:33.212143    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-r6hs9" podStartSLOduration=67.787444665 podStartE2EDuration="1m9.212120656s" totalImagesPullingTime="1.424675991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:33.211604682 +0000 UTC m=+68.984844713" watchObservedRunningTime="2026-09-11 16:18:33.212120656 +0000 UTC m=+68.985360686"
Sep 11 16:18:34.934718 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.934677    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:18:34.936897 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.936875    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:34.940229 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.940209    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 16:18:34.941374 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941332    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5nx5t\""
Sep 11 16:18:34.941374 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941332    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 16:18:34.941374 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941338    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 16:18:34.941647 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941424    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 16:18:34.941647 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941498    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 16:18:34.941647 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941517    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 16:18:34.941797 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.941770    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 16:18:34.948011 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.947988    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 16:18:34.953108 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:34.953082    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:18:35.103631 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103588    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103711    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmsjz\" (UniqueName: \"kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103794    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103956 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103819    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103956 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103847    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.103956 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.103867    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204560 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmsjz\" (UniqueName: \"kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204596    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204656    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204920 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204723    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.204920 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.204749    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.206162 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.206131    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.207467 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.207418    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.207634 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.207611    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.208108 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.208083    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.209513 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.209490    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.209910 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.209888    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.226379 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.226353    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmsjz\" (UniqueName: \"kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz\") pod \"console-7bc48df577-jb4x7\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") " pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.248921 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.248894    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:35.536035 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:35.533513    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:18:35.538837 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:35.538810    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07a7b885_b115_4b16_be15_175fbb53744c.slice/crio-7a242586e5dcd68e0669cd30d1584a6e88604615fc49c39450ef72697a990f2c WatchSource:0}: Error finding container 7a242586e5dcd68e0669cd30d1584a6e88604615fc49c39450ef72697a990f2c: Status 404 returned error can't find the container with id 7a242586e5dcd68e0669cd30d1584a6e88604615fc49c39450ef72697a990f2c
Sep 11 16:18:36.205009 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.204961    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ft8q5" event={"ID":"d2003fb4-89c4-4059-8096-cc104438d87c","Type":"ContainerStarted","Data":"8402999366a9f4f941d9added6d4907806f79658c77ed8fccd6f0bab1a862c80"}
Sep 11 16:18:36.206728 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.206696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xp82l" event={"ID":"6348cc3f-d5db-44a6-91dc-bdc28ce89b6d","Type":"ContainerStarted","Data":"2436de3a1ebae97c14fc9c60932c53ad7503a52c2ee464a10a1bc1760e431b00"}
Sep 11 16:18:36.206872 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.206761    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:18:36.208161 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.208139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc48df577-jb4x7" event={"ID":"07a7b885-b115-4b16-be15-175fbb53744c","Type":"ContainerStarted","Data":"7a242586e5dcd68e0669cd30d1584a6e88604615fc49c39450ef72697a990f2c"}
Sep 11 16:18:36.233048 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.232981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ft8q5" podStartSLOduration=66.609684217 podStartE2EDuration="1m11.232958324s" totalImagesPullingTime="4.623274107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 16:17:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:36.232550933 +0000 UTC m=+72.005790972" watchObservedRunningTime="2026-09-11 16:18:36.232958324 +0000 UTC m=+72.006198356"
Sep 11 16:18:36.261900 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:36.261794    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xp82l" podStartSLOduration=67.640350247 podStartE2EDuration="1m12.261771067s" totalImagesPullingTime="4.62142082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:36.260994905 +0000 UTC m=+72.034234937" watchObservedRunningTime="2026-09-11 16:18:36.261771067 +0000 UTC m=+72.035011099"
Sep 11 16:18:38.217608 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:38.217560    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc48df577-jb4x7" event={"ID":"07a7b885-b115-4b16-be15-175fbb53744c","Type":"ContainerStarted","Data":"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"}
Sep 11 16:18:38.255757 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:38.255697    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bc48df577-jb4x7" podStartSLOduration=1.675986057 podStartE2EDuration="4.255659598s" totalImagesPullingTime="2.579673541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:38.254595904 +0000 UTC m=+74.027835934" watchObservedRunningTime="2026-09-11 16:18:38.255659598 +0000 UTC m=+74.028899628"
Sep 11 16:18:45.249013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:45.248978    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:45.249468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:45.249058    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:45.253992 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:45.253969    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:46.252628 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:46.252599    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:18:48.150420 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:48.150390    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cbcdf84f-g8gp4"
Sep 11 16:18:49.829287 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.829242    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pcl2g"]
Sep 11 16:18:49.832368 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.832344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:49.835577 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.835536    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 16:18:49.835808 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.835781    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 16:18:49.836005 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.835987    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 16:18:49.836140 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.835631    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 16:18:49.836140 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.835678    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 16:18:49.836320 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.836306    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 16:18:49.836816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:49.836801    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wc8kf\""
Sep 11 16:18:50.017549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017512    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x4js\" (UniqueName: \"kubernetes.io/projected/a5e54a6c-de38-4b8a-8d40-c96668e58957-kube-api-access-9x4js\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017558    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017595    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-root\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017619    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-textfile\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-wtmp\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017762    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-sys\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017791    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.017835 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.017835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-metrics-client-ca\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119152 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-root\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119155    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-textfile\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119200    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119238    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-wtmp\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119277    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-sys\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119353 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-metrics-client-ca\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119674 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x4js\" (UniqueName: \"kubernetes.io/projected/a5e54a6c-de38-4b8a-8d40-c96668e58957-kube-api-access-9x4js\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119674 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.119440    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.119674 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:50.119634    2579 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 11 16:18:50.119816 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:18:50.119752    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls podName:a5e54a6c-de38-4b8a-8d40-c96668e58957 nodeName:}" failed. No retries permitted until 2026-09-11 16:18:50.619730266 +0000 UTC m=+86.392970282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls") pod "node-exporter-pcl2g" (UID: "a5e54a6c-de38-4b8a-8d40-c96668e58957") : secret "node-exporter-tls" not found
Sep 11 16:18:50.120128 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.120094    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-root\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.120616 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.120595    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-textfile\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.121650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.121627    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-wtmp\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.121752 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.121704    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e54a6c-de38-4b8a-8d40-c96668e58957-sys\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.122347 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.122327    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.122893 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.122874    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e54a6c-de38-4b8a-8d40-c96668e58957-metrics-client-ca\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.125483 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.125437    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.131740 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.131711    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x4js\" (UniqueName: \"kubernetes.io/projected/a5e54a6c-de38-4b8a-8d40-c96668e58957-kube-api-access-9x4js\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.623330 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.623290    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.625908 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.625884    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e54a6c-de38-4b8a-8d40-c96668e58957-node-exporter-tls\") pod \"node-exporter-pcl2g\" (UID: \"a5e54a6c-de38-4b8a-8d40-c96668e58957\") " pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.744557 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:50.744519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pcl2g"
Sep 11 16:18:50.755894 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:18:50.755862    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e54a6c_de38_4b8a_8d40_c96668e58957.slice/crio-6560dd4082a59f3edc6c6b6a12991883e7425d48e0d6e79c0918ab1ea5c67d2c WatchSource:0}: Error finding container 6560dd4082a59f3edc6c6b6a12991883e7425d48e0d6e79c0918ab1ea5c67d2c: Status 404 returned error can't find the container with id 6560dd4082a59f3edc6c6b6a12991883e7425d48e0d6e79c0918ab1ea5c67d2c
Sep 11 16:18:51.266972 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:51.266935    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcl2g" event={"ID":"a5e54a6c-de38-4b8a-8d40-c96668e58957","Type":"ContainerStarted","Data":"6560dd4082a59f3edc6c6b6a12991883e7425d48e0d6e79c0918ab1ea5c67d2c"}
Sep 11 16:18:52.272515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:52.272482    2579 generic.go:356] "Generic (PLEG): container finished" podID="a5e54a6c-de38-4b8a-8d40-c96668e58957" containerID="8149ef66e47ec1e9334afa1d023e84b9a6b2a3c50fe23e578335262aa752365d" exitCode=0
Sep 11 16:18:52.272925 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:52.272548    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcl2g" event={"ID":"a5e54a6c-de38-4b8a-8d40-c96668e58957","Type":"ContainerDied","Data":"8149ef66e47ec1e9334afa1d023e84b9a6b2a3c50fe23e578335262aa752365d"}
Sep 11 16:18:53.277771 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:53.277730    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcl2g" event={"ID":"a5e54a6c-de38-4b8a-8d40-c96668e58957","Type":"ContainerStarted","Data":"fa61e98eddca5d1ff7b2bd09c26a12698b3db9851cc1027f3b14f2e22949c7fb"}
Sep 11 16:18:53.278166 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:53.277803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcl2g" event={"ID":"a5e54a6c-de38-4b8a-8d40-c96668e58957","Type":"ContainerStarted","Data":"642c40c5e04496aba5980a10bdbb34bb483a780871260e782f3662280c78abfc"}
Sep 11 16:18:53.302559 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:18:53.302507    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pcl2g" podStartSLOduration=3.187177966 podStartE2EDuration="4.302487833s" totalImagesPullingTime="1.041506043s" totalInitContainerRuntime="73.803824ms" isStatefulPod=true podCreationTimestamp="2026-09-11 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:18:53.300561352 +0000 UTC m=+89.073801384" watchObservedRunningTime="2026-09-11 16:18:53.302487833 +0000 UTC m=+89.075727864"
Sep 11 16:19:05.653515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:05.653484    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:19:07.215527 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:07.215499    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xp82l"
Sep 11 16:19:30.676471 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:30.676408    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bc48df577-jb4x7" podUID="07a7b885-b115-4b16-be15-175fbb53744c" containerName="console" containerID="cri-o://775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4" gracePeriod=15
Sep 11 16:19:30.913811 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:30.913790    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:19:31.024324 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024254    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024324 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024289    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmsjz\" (UniqueName: \"kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024324 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024312    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024599 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024349    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024599 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024369    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024599 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024413    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024599 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert\") pod \"07a7b885-b115-4b16-be15-175fbb53744c\" (UID: \"07a7b885-b115-4b16-be15-175fbb53744c\") "
Sep 11 16:19:31.024876 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024828    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:31.024876 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024849    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca" (OuterVolumeSpecName: "service-ca") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:31.025003 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024964    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config" (OuterVolumeSpecName: "console-config") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:31.025130 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.024967    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:19:31.026719 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.026683    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:19:31.026816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.026686    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz" (OuterVolumeSpecName: "kube-api-access-qmsjz") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "kube-api-access-qmsjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:19:31.026816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.026738    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07a7b885-b115-4b16-be15-175fbb53744c" (UID: "07a7b885-b115-4b16-be15-175fbb53744c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:19:31.125490 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125423    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-oauth-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125490 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125483    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07a7b885-b115-4b16-be15-175fbb53744c-console-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125490 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125494    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-console-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125490 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125503    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-oauth-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125513    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-trusted-ca-bundle\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125523    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmsjz\" (UniqueName: \"kubernetes.io/projected/07a7b885-b115-4b16-be15-175fbb53744c-kube-api-access-qmsjz\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.125742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.125531    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07a7b885-b115-4b16-be15-175fbb53744c-service-ca\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:19:31.408500 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.408438    2579 generic.go:356] "Generic (PLEG): container finished" podID="07a7b885-b115-4b16-be15-175fbb53744c" containerID="775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4" exitCode=0
Sep 11 16:19:31.408500 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.408495    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc48df577-jb4x7"
Sep 11 16:19:31.408736 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.408549    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc48df577-jb4x7" event={"ID":"07a7b885-b115-4b16-be15-175fbb53744c","Type":"ContainerDied","Data":"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"}
Sep 11 16:19:31.408736 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.408593    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc48df577-jb4x7" event={"ID":"07a7b885-b115-4b16-be15-175fbb53744c","Type":"ContainerDied","Data":"7a242586e5dcd68e0669cd30d1584a6e88604615fc49c39450ef72697a990f2c"}
Sep 11 16:19:31.408736 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.408612    2579 scope.go:117] "RemoveContainer" containerID="775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"
Sep 11 16:19:31.417053 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.417035    2579 scope.go:117] "RemoveContainer" containerID="775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"
Sep 11 16:19:31.417312 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:19:31.417294    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4\": container with ID starting with 775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4 not found: ID does not exist" containerID="775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"
Sep 11 16:19:31.417351 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.417323    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4"} err="failed to get container status \"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4\": rpc error: code = NotFound desc = could not find container \"775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4\": container with ID starting with 775a996e91a012b543b1da7055617459bf41c677fe3849be3e7d10b6d54b1dc4 not found: ID does not exist"
Sep 11 16:19:31.429287 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.429264    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:19:31.433600 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:31.433579    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bc48df577-jb4x7"]
Sep 11 16:19:32.749079 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:19:32.749043    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a7b885-b115-4b16-be15-175fbb53744c" path="/var/lib/kubelet/pods/07a7b885-b115-4b16-be15-175fbb53744c/volumes"
Sep 11 16:20:18.119655 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.119572    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:18.120124 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.119844    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a7b885-b115-4b16-be15-175fbb53744c" containerName="console"
Sep 11 16:20:18.120124 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.119855    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a7b885-b115-4b16-be15-175fbb53744c" containerName="console"
Sep 11 16:20:18.120124 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.119902    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a7b885-b115-4b16-be15-175fbb53744c" containerName="console"
Sep 11 16:20:18.121697 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.121679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.125365 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125340    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 11 16:20:18.125484 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125340    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 16:20:18.125657 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125626    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 11 16:20:18.125657 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125640    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 16:20:18.125657 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125654    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 11 16:20:18.125862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125704    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 11 16:20:18.125862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125704    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 11 16:20:18.125862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.125642    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5nx5t\""
Sep 11 16:20:18.130825 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.130808    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 11 16:20:18.137587 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.137566    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:18.248501 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248501 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248498    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248530    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248594    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248610    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.248724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.248680    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phkhc\" (UniqueName: \"kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349237    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phkhc\" (UniqueName: \"kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349277 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349275    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349520 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349520 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349346    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349520 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349651 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349533    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.349651 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.349579    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.350226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.350205    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.350325 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.350262    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.350385 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.350365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.350561 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.350540    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.352074 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.352054    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.352123 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.352088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.357775 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.357757    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phkhc\" (UniqueName: \"kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc\") pod \"console-859fdd89f-qdcxd\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") " pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.431515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.431403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:18.702718 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:18.702643    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:18.706772 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:20:18.706743    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod178e475b_cd3f_4827_9114_23ecf5722a63.slice/crio-d908b787024a68e9d9ec75fbb17395bb5b58b216c2c4be3143f66ed780e24d80 WatchSource:0}: Error finding container d908b787024a68e9d9ec75fbb17395bb5b58b216c2c4be3143f66ed780e24d80: Status 404 returned error can't find the container with id d908b787024a68e9d9ec75fbb17395bb5b58b216c2c4be3143f66ed780e24d80
Sep 11 16:20:19.575968 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:19.575924    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859fdd89f-qdcxd" event={"ID":"178e475b-cd3f-4827-9114-23ecf5722a63","Type":"ContainerStarted","Data":"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"}
Sep 11 16:20:19.576506 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:19.575989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859fdd89f-qdcxd" event={"ID":"178e475b-cd3f-4827-9114-23ecf5722a63","Type":"ContainerStarted","Data":"d908b787024a68e9d9ec75fbb17395bb5b58b216c2c4be3143f66ed780e24d80"}
Sep 11 16:20:19.596421 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:19.596367    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-859fdd89f-qdcxd" podStartSLOduration=1.596349797 podStartE2EDuration="1.596349797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:20:19.59557745 +0000 UTC m=+175.368817484" watchObservedRunningTime="2026-09-11 16:20:19.596349797 +0000 UTC m=+175.369589828"
Sep 11 16:20:28.432240 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.432200    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:28.432685 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.432270    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:28.438124 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.438098    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:28.609935 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.609905    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:28.614977 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.614952    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:28.659810 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.659776    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:20:28.662785 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.662769    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.673495 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.673469    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:20:28.827214 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827043    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827214 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827169    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827308    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827357    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpk2v\" (UniqueName: \"kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827395    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.827553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.827417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928206    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928246 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928482 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928270    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928482 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928482 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928482 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpk2v\" (UniqueName: \"kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.928482 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.928425    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.929159 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.929125    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.929274 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.929243    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.929331 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.929302    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.929422 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.929397    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.931073 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.931057    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.931131 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.931114    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.937987 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.937964    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpk2v\" (UniqueName: \"kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v\") pod \"console-669b645678-fp894\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") " pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:28.971921 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:28.971883    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:29.244560 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:29.244525    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:20:29.247538 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:20:29.247507    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5410b330_c3e2_4de9_a0b5_f48a1ec2f573.slice/crio-e2f365ddba6ef6df1ebc3f518a383f0b965c1e515d414cb105afb7d0d5da494e WatchSource:0}: Error finding container e2f365ddba6ef6df1ebc3f518a383f0b965c1e515d414cb105afb7d0d5da494e: Status 404 returned error can't find the container with id e2f365ddba6ef6df1ebc3f518a383f0b965c1e515d414cb105afb7d0d5da494e
Sep 11 16:20:29.611473 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:29.611333    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669b645678-fp894" event={"ID":"5410b330-c3e2-4de9-a0b5-f48a1ec2f573","Type":"ContainerStarted","Data":"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"}
Sep 11 16:20:29.611860 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:29.611471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669b645678-fp894" event={"ID":"5410b330-c3e2-4de9-a0b5-f48a1ec2f573","Type":"ContainerStarted","Data":"e2f365ddba6ef6df1ebc3f518a383f0b965c1e515d414cb105afb7d0d5da494e"}
Sep 11 16:20:29.631885 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:29.631832    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-669b645678-fp894" podStartSLOduration=1.631813567 podStartE2EDuration="1.631813567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:20:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:20:29.631001756 +0000 UTC m=+185.404241787" watchObservedRunningTime="2026-09-11 16:20:29.631813567 +0000 UTC m=+185.405053597"
Sep 11 16:20:38.972520 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:38.972485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:38.972965 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:38.972540    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:38.977303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:38.977284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:39.647523 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:39.647496    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:20:55.633714 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.633652    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-859fdd89f-qdcxd" podUID="178e475b-cd3f-4827-9114-23ecf5722a63" containerName="console" containerID="cri-o://535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889" gracePeriod=15
Sep 11 16:20:55.874049 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.874024    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:55.930019 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.929925    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930019 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.929970    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phkhc\" (UniqueName: \"kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930019 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.929993    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930019 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930015    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930350 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930031    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930350 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930048    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930350 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930068    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca\") pod \"178e475b-cd3f-4827-9114-23ecf5722a63\" (UID: \"178e475b-cd3f-4827-9114-23ecf5722a63\") "
Sep 11 16:20:55.930549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930386    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:20:55.930604 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930547    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:20:55.930604 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930505    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config" (OuterVolumeSpecName: "console-config") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:20:55.930710 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.930631    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca" (OuterVolumeSpecName: "service-ca") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:20:55.932427 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.932391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc" (OuterVolumeSpecName: "kube-api-access-phkhc") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "kube-api-access-phkhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:20:55.932581 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.932417    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:20:55.932581 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:55.932487    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "178e475b-cd3f-4827-9114-23ecf5722a63" (UID: "178e475b-cd3f-4827-9114-23ecf5722a63"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:20:56.031226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031174    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-oauth-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031218    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phkhc\" (UniqueName: \"kubernetes.io/projected/178e475b-cd3f-4827-9114-23ecf5722a63-kube-api-access-phkhc\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031228    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-trusted-ca-bundle\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031236    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031226 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031244    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/178e475b-cd3f-4827-9114-23ecf5722a63-console-oauth-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031570 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031252    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-console-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.031570 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.031260    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/178e475b-cd3f-4827-9114-23ecf5722a63-service-ca\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:20:56.704183 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.704143    2579 generic.go:356] "Generic (PLEG): container finished" podID="178e475b-cd3f-4827-9114-23ecf5722a63" containerID="535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889" exitCode=0
Sep 11 16:20:56.704694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.704200    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-859fdd89f-qdcxd"
Sep 11 16:20:56.704694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.704201    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859fdd89f-qdcxd" event={"ID":"178e475b-cd3f-4827-9114-23ecf5722a63","Type":"ContainerDied","Data":"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"}
Sep 11 16:20:56.704694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.704324    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-859fdd89f-qdcxd" event={"ID":"178e475b-cd3f-4827-9114-23ecf5722a63","Type":"ContainerDied","Data":"d908b787024a68e9d9ec75fbb17395bb5b58b216c2c4be3143f66ed780e24d80"}
Sep 11 16:20:56.704694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.704353    2579 scope.go:117] "RemoveContainer" containerID="535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"
Sep 11 16:20:56.712776 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.712760    2579 scope.go:117] "RemoveContainer" containerID="535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"
Sep 11 16:20:56.713102 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:20:56.713084    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889\": container with ID starting with 535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889 not found: ID does not exist" containerID="535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"
Sep 11 16:20:56.713206 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.713112    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889"} err="failed to get container status \"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889\": rpc error: code = NotFound desc = could not find container \"535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889\": container with ID starting with 535c39b450a7167d50733a305c4fc15f5a089f51e22917e2aaa527c6d1747889 not found: ID does not exist"
Sep 11 16:20:56.725124 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.725095    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:56.729719 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.729691    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-859fdd89f-qdcxd"]
Sep 11 16:20:56.749781 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:20:56.749753    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="178e475b-cd3f-4827-9114-23ecf5722a63" path="/var/lib/kubelet/pods/178e475b-cd3f-4827-9114-23ecf5722a63/volumes"
Sep 11 16:22:24.619791 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:24.619762    2579 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 16:22:40.574263 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.574228    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-wppnm"]
Sep 11 16:22:40.575654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.574528    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="178e475b-cd3f-4827-9114-23ecf5722a63" containerName="console"
Sep 11 16:22:40.575654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.574541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="178e475b-cd3f-4827-9114-23ecf5722a63" containerName="console"
Sep 11 16:22:40.575654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.574593    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="178e475b-cd3f-4827-9114-23ecf5722a63" containerName="console"
Sep 11 16:22:40.576182 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.576167    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:40.579572 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.579549    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-hh4mp\""
Sep 11 16:22:40.579790 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.579773    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 11 16:22:40.579872 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.579812    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 11 16:22:40.580643 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.580625    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 11 16:22:40.580717 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.580676    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 11 16:22:40.597528 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.597503    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-wppnm"]
Sep 11 16:22:40.689051 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.689018    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2hpl\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-kube-api-access-l2hpl\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:40.689216 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.689086    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:40.789880 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.789850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2hpl\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-kube-api-access-l2hpl\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:40.790075 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.789918    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:40.790075 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:22:40.790048    2579 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Sep 11 16:22:40.790075 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:22:40.790071    2579 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-6c4d864b86-wppnm: secret "keda-admission-webhooks-certs" not found
Sep 11 16:22:40.790240 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:22:40.790139    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates podName:386e5afa-9503-44c9-abd3-ea0eefb7a52b nodeName:}" failed. No retries permitted until 2026-09-11 16:22:41.290118741 +0000 UTC m=+317.063358766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates") pod "keda-admission-6c4d864b86-wppnm" (UID: "386e5afa-9503-44c9-abd3-ea0eefb7a52b") : secret "keda-admission-webhooks-certs" not found
Sep 11 16:22:40.805196 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:40.805171    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2hpl\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-kube-api-access-l2hpl\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:41.292880 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:41.292845    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:41.295553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:41.295522    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/386e5afa-9503-44c9-abd3-ea0eefb7a52b-certificates\") pod \"keda-admission-6c4d864b86-wppnm\" (UID: \"386e5afa-9503-44c9-abd3-ea0eefb7a52b\") " pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:41.486655 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:41.486600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:41.772001 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:41.771970    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-wppnm"]
Sep 11 16:22:41.775188 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:22:41.775160    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386e5afa_9503_44c9_abd3_ea0eefb7a52b.slice/crio-70e832b68a9539faef43d87e6f743834ea24c46f7fda3d10897e35d383d78293 WatchSource:0}: Error finding container 70e832b68a9539faef43d87e6f743834ea24c46f7fda3d10897e35d383d78293: Status 404 returned error can't find the container with id 70e832b68a9539faef43d87e6f743834ea24c46f7fda3d10897e35d383d78293
Sep 11 16:22:41.776549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:41.776530    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 16:22:42.060060 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:42.059970    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-wppnm" event={"ID":"386e5afa-9503-44c9-abd3-ea0eefb7a52b","Type":"ContainerStarted","Data":"70e832b68a9539faef43d87e6f743834ea24c46f7fda3d10897e35d383d78293"}
Sep 11 16:22:46.075824 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:46.075786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-wppnm" event={"ID":"386e5afa-9503-44c9-abd3-ea0eefb7a52b","Type":"ContainerStarted","Data":"1672cbaf7d2fb1cbcbb94420834ac4592ce97680c1a68369a1bd63a1949b959e"}
Sep 11 16:22:46.076316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:46.075949    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:22:46.094635 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:22:46.094587    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-wppnm" podStartSLOduration=2.546362347 podStartE2EDuration="6.094569333s" totalImagesPullingTime="3.548206986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:22:46.094191998 +0000 UTC m=+321.867432029" watchObservedRunningTime="2026-09-11 16:22:46.094569333 +0000 UTC m=+321.867809365"
Sep 11 16:23:07.082043 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:07.082000    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-wppnm"
Sep 11 16:23:33.161416 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.161330    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d44f4bcfc-xvx62"]
Sep 11 16:23:33.168974 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.168952    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.176173 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.175962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d44f4bcfc-xvx62"]
Sep 11 16:23:33.180970 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.180944    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-oauth-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181096 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.180978    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-trusted-ca-bundle\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181096 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.180998    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-oauth-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181096 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.181031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181199 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.181112    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181233 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.181203    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-service-ca\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.181233 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.181228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chggf\" (UniqueName: \"kubernetes.io/projected/43c7f3e7-076d-410d-a329-31da4b0d75b1-kube-api-access-chggf\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282211 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282177    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-oauth-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282211 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282215    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-trusted-ca-bundle\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-oauth-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282266    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282307    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-service-ca\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.282489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.282332    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chggf\" (UniqueName: \"kubernetes.io/projected/43c7f3e7-076d-410d-a329-31da4b0d75b1-kube-api-access-chggf\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.283055 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.283020    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-oauth-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.283055 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.283046    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.283381 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.283359    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-trusted-ca-bundle\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.283419 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.283402    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/43c7f3e7-076d-410d-a329-31da4b0d75b1-service-ca\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.285035 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.285014    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-oauth-config\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.285135 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.285088    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/43c7f3e7-076d-410d-a329-31da4b0d75b1-console-serving-cert\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.295658 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.295631    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chggf\" (UniqueName: \"kubernetes.io/projected/43c7f3e7-076d-410d-a329-31da4b0d75b1-kube-api-access-chggf\") pod \"console-6d44f4bcfc-xvx62\" (UID: \"43c7f3e7-076d-410d-a329-31da4b0d75b1\") " pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.479338 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.479243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:33.751501 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:33.751405    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d44f4bcfc-xvx62"]
Sep 11 16:23:33.754428 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:23:33.754400    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c7f3e7_076d_410d_a329_31da4b0d75b1.slice/crio-96ec056259e1c7fd1cc08291228212f0f8501fb0a9a22dfe7c0830196d882bcc WatchSource:0}: Error finding container 96ec056259e1c7fd1cc08291228212f0f8501fb0a9a22dfe7c0830196d882bcc: Status 404 returned error can't find the container with id 96ec056259e1c7fd1cc08291228212f0f8501fb0a9a22dfe7c0830196d882bcc
Sep 11 16:23:34.248782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:34.248742    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44f4bcfc-xvx62" event={"ID":"43c7f3e7-076d-410d-a329-31da4b0d75b1","Type":"ContainerStarted","Data":"3d2e467427e2ca2fe3b857b48649318fa510b420cc7e7a3b3439fad9966e573e"}
Sep 11 16:23:34.249207 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:34.248800    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d44f4bcfc-xvx62" event={"ID":"43c7f3e7-076d-410d-a329-31da4b0d75b1","Type":"ContainerStarted","Data":"96ec056259e1c7fd1cc08291228212f0f8501fb0a9a22dfe7c0830196d882bcc"}
Sep 11 16:23:34.267739 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:34.267687    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d44f4bcfc-xvx62" podStartSLOduration=1.267668796 podStartE2EDuration="1.267668796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:23:34.266612947 +0000 UTC m=+370.039852977" watchObservedRunningTime="2026-09-11 16:23:34.267668796 +0000 UTC m=+370.040908828"
Sep 11 16:23:43.480313 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:43.480257    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:43.480750 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:43.480335    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:43.485146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:43.485120    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:44.286542 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:44.286515    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d44f4bcfc-xvx62"
Sep 11 16:23:44.337193 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:44.337150    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:23:58.318001 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.317969    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-b8j85"]
Sep 11 16:23:58.321098 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.321078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.323723 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.323703    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 11 16:23:58.323820 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.323741    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 11 16:23:58.324665 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.324649    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-tc4d7\""
Sep 11 16:23:58.324749 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.324665    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 11 16:23:58.333560 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.333536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-b8j85"]
Sep 11 16:23:58.374464 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.374420    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/69e8fc45-2e3a-4441-871f-86e47fef596c-data\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.374584 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.374471    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcjhf\" (UniqueName: \"kubernetes.io/projected/69e8fc45-2e3a-4441-871f-86e47fef596c-kube-api-access-mcjhf\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.475243 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.475205    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/69e8fc45-2e3a-4441-871f-86e47fef596c-data\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.475243 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.475243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcjhf\" (UniqueName: \"kubernetes.io/projected/69e8fc45-2e3a-4441-871f-86e47fef596c-kube-api-access-mcjhf\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.475696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.475677    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/69e8fc45-2e3a-4441-871f-86e47fef596c-data\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.483849 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.483829    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcjhf\" (UniqueName: \"kubernetes.io/projected/69e8fc45-2e3a-4441-871f-86e47fef596c-kube-api-access-mcjhf\") pod \"seaweedfs-64568bcd49-b8j85\" (UID: \"69e8fc45-2e3a-4441-871f-86e47fef596c\") " pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.630653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.630626    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:23:58.899280 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:58.899211    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-b8j85"]
Sep 11 16:23:58.902725 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:23:58.902697    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69e8fc45_2e3a_4441_871f_86e47fef596c.slice/crio-08bbb62dfebddb65edf1f12a30060c56ab78e02cabfe87ebe8558870d8e67866 WatchSource:0}: Error finding container 08bbb62dfebddb65edf1f12a30060c56ab78e02cabfe87ebe8558870d8e67866: Status 404 returned error can't find the container with id 08bbb62dfebddb65edf1f12a30060c56ab78e02cabfe87ebe8558870d8e67866
Sep 11 16:23:59.339138 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:23:59.339058    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-b8j85" event={"ID":"69e8fc45-2e3a-4441-871f-86e47fef596c","Type":"ContainerStarted","Data":"08bbb62dfebddb65edf1f12a30060c56ab78e02cabfe87ebe8558870d8e67866"}
Sep 11 16:24:02.352925 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:02.352854    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-b8j85" event={"ID":"69e8fc45-2e3a-4441-871f-86e47fef596c","Type":"ContainerStarted","Data":"0a9a3bad9666453b31db52df8cf33d131cfe3d8aac5987b642fa4641e91fb916"}
Sep 11 16:24:02.353296 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:02.353001    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:24:02.368235 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:02.368182    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-b8j85" podStartSLOduration=1.639623909 podStartE2EDuration="4.368170862s" totalImagesPullingTime="2.728546953s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:23:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:24:02.368046218 +0000 UTC m=+398.141286255" watchObservedRunningTime="2026-09-11 16:24:02.368170862 +0000 UTC m=+398.141410892"
Sep 11 16:24:08.358852 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:08.358820    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-b8j85"
Sep 11 16:24:09.356549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.356504    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-669b645678-fp894" podUID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" containerName="console" containerID="cri-o://cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558" gracePeriod=15
Sep 11 16:24:09.591016 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.590989    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:24:09.678577 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678536    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpk2v\" (UniqueName: \"kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678577 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678848 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678601    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678848 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678645    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678848 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678848 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678706    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.678848 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.678758    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca\") pod \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\" (UID: \"5410b330-c3e2-4de9-a0b5-f48a1ec2f573\") "
Sep 11 16:24:09.679198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.679170    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config" (OuterVolumeSpecName: "console-config") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:24:09.679265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.679195    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca" (OuterVolumeSpecName: "service-ca") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:24:09.679265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.679196    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:24:09.679265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.679249    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:24:09.681072 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.681037    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:24:09.681399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.681380    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v" (OuterVolumeSpecName: "kube-api-access-rpk2v") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "kube-api-access-rpk2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:24:09.681542 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.681394    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5410b330-c3e2-4de9-a0b5-f48a1ec2f573" (UID: "5410b330-c3e2-4de9-a0b5-f48a1ec2f573"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:24:09.780291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780236    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-trusted-ca-bundle\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780282    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-service-ca\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780291    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpk2v\" (UniqueName: \"kubernetes.io/projected/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-kube-api-access-rpk2v\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780300    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780310    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-oauth-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780319    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-console-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:09.780654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:09.780327    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5410b330-c3e2-4de9-a0b5-f48a1ec2f573-oauth-serving-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:24:10.387996 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.387956    2579 generic.go:356] "Generic (PLEG): container finished" podID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" containerID="cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558" exitCode=0
Sep 11 16:24:10.387996 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.387998    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669b645678-fp894" event={"ID":"5410b330-c3e2-4de9-a0b5-f48a1ec2f573","Type":"ContainerDied","Data":"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"}
Sep 11 16:24:10.388261 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.388014    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-669b645678-fp894"
Sep 11 16:24:10.388261 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.388065    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-669b645678-fp894" event={"ID":"5410b330-c3e2-4de9-a0b5-f48a1ec2f573","Type":"ContainerDied","Data":"e2f365ddba6ef6df1ebc3f518a383f0b965c1e515d414cb105afb7d0d5da494e"}
Sep 11 16:24:10.388261 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.388085    2579 scope.go:117] "RemoveContainer" containerID="cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"
Sep 11 16:24:10.396840 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.396817    2579 scope.go:117] "RemoveContainer" containerID="cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"
Sep 11 16:24:10.397150 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:24:10.397129    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558\": container with ID starting with cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558 not found: ID does not exist" containerID="cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"
Sep 11 16:24:10.397225 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.397165    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558"} err="failed to get container status \"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558\": rpc error: code = NotFound desc = could not find container \"cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558\": container with ID starting with cb806b8618e3badf2fdb8e19f8ef2691b9e83e3ac891d11c64c61bfa31191558 not found: ID does not exist"
Sep 11 16:24:10.409844 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.409812    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:24:10.414202 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.414170    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-669b645678-fp894"]
Sep 11 16:24:10.750519 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:24:10.750416    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" path="/var/lib/kubelet/pods/5410b330-c3e2-4de9-a0b5-f48a1ec2f573/volumes"
Sep 11 16:25:32.218405 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.218368    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-68ptc"]
Sep 11 16:25:32.218892 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.218712    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" containerName="console"
Sep 11 16:25:32.218892 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.218730    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" containerName="console"
Sep 11 16:25:32.218892 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.218806    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="5410b330-c3e2-4de9-a0b5-f48a1ec2f573" containerName="console"
Sep 11 16:25:32.221881 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.221861    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.224578 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.224559    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"hf-token\""
Sep 11 16:25:32.229896 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.229872    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-68ptc"]
Sep 11 16:25:32.295940 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.295897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td7rf\" (UniqueName: \"kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.296120 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.295964    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.397234 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.397191    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td7rf\" (UniqueName: \"kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.397407 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.397267    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.397696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.397678    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.406802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.406771    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td7rf\" (UniqueName: \"kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf\") pod \"s3-init-68ptc\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") " pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.531046 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.530949    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-68ptc"
Sep 11 16:25:32.800411 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:32.800335    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-68ptc"]
Sep 11 16:25:32.803632 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:25:32.803604    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode17e0022_1827_408f_8f59_21dbd6d68acf.slice/crio-1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc WatchSource:0}: Error finding container 1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc: Status 404 returned error can't find the container with id 1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc
Sep 11 16:25:33.674953 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:33.674906    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerStarted","Data":"1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc"}
Sep 11 16:25:36.688653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:36.688620    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerStarted","Data":"27431ff02a2d84b4dc695fa2ac60391fd5c99c5312c98738b37afd1f290d22e9"}
Sep 11 16:25:38.696999 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:38.696963    2579 generic.go:356] "Generic (PLEG): container finished" podID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerID="27431ff02a2d84b4dc695fa2ac60391fd5c99c5312c98738b37afd1f290d22e9" exitCode=0
Sep 11 16:25:38.697370 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:38.697037    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerDied","Data":"27431ff02a2d84b4dc695fa2ac60391fd5c99c5312c98738b37afd1f290d22e9"}
Sep 11 16:25:43.719462 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:43.719415    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerStarted","Data":"2ce2e793b5f85333af26b313e6a730a4b0b1d7ef7ec013aec7787eddf0fb2a98"}
Sep 11 16:25:43.736917 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:25:43.736865    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-68ptc" podStartSLOduration=2.303855093 podStartE2EDuration="11.736849498s" totalImagesPullingTime="7.710548012s" totalInitContainerRuntime="1.722446393s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:25:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:25:43.73512403 +0000 UTC m=+499.508364075" watchObservedRunningTime="2026-09-11 16:25:43.736849498 +0000 UTC m=+499.510089528"
Sep 11 16:26:47.944215 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:47.944182    2579 generic.go:356] "Generic (PLEG): container finished" podID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerID="2ce2e793b5f85333af26b313e6a730a4b0b1d7ef7ec013aec7787eddf0fb2a98" exitCode=0
Sep 11 16:26:47.944652 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:47.944256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerDied","Data":"2ce2e793b5f85333af26b313e6a730a4b0b1d7ef7ec013aec7787eddf0fb2a98"}
Sep 11 16:26:49.066077 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.066052    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-68ptc"
Sep 11 16:26:49.193701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.193669    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache\") pod \"e17e0022-1827-408f-8f59-21dbd6d68acf\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") "
Sep 11 16:26:49.193863 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.193725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td7rf\" (UniqueName: \"kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf\") pod \"e17e0022-1827-408f-8f59-21dbd6d68acf\" (UID: \"e17e0022-1827-408f-8f59-21dbd6d68acf\") "
Sep 11 16:26:49.196015 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.195990    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf" (OuterVolumeSpecName: "kube-api-access-td7rf") pod "e17e0022-1827-408f-8f59-21dbd6d68acf" (UID: "e17e0022-1827-408f-8f59-21dbd6d68acf"). InnerVolumeSpecName "kube-api-access-td7rf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:26:49.212000 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.211968    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e17e0022-1827-408f-8f59-21dbd6d68acf" (UID: "e17e0022-1827-408f-8f59-21dbd6d68acf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:26:49.294531 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.294499    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td7rf\" (UniqueName: \"kubernetes.io/projected/e17e0022-1827-408f-8f59-21dbd6d68acf-kube-api-access-td7rf\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:26:49.294531 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.294528    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e17e0022-1827-408f-8f59-21dbd6d68acf-model-cache\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:26:49.952976 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.952937    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-68ptc"
Sep 11 16:26:49.952976 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.952944    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-68ptc" event={"ID":"e17e0022-1827-408f-8f59-21dbd6d68acf","Type":"ContainerDied","Data":"1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc"}
Sep 11 16:26:49.953231 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:26:49.952986    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e4aafd6ca7e2783404bb214082178d22d0a1a5145c3f80b0db761b22d7dc4bc"
Sep 11 16:27:01.653271 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653234    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:27:01.653696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653613    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerName="s3-init"
Sep 11 16:27:01.653696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653628    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerName="s3-init"
Sep 11 16:27:01.653696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653650    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerName="download-hf-model"
Sep 11 16:27:01.653696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653660    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerName="download-hf-model"
Sep 11 16:27:01.653824 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.653719    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e17e0022-1827-408f-8f59-21dbd6d68acf" containerName="s3-init"
Sep 11 16:27:01.657336 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.657315    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.660827 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.660806    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\""
Sep 11 16:27:01.660926 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.660907    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 11 16:27:01.661076 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.661061    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-mmq9q\""
Sep 11 16:27:01.661121 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.661098    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 11 16:27:01.661321 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.661297    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\""
Sep 11 16:27:01.661731 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.661705    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 11 16:27:01.661731 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.661722    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 11 16:27:01.672947 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.672924    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:27:01.752078 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.752040    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:27:01.755074 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.755057    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.757464 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.757423    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8975a-kube-rbac-proxy-sar-config\""
Sep 11 16:27:01.757464 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.757442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8975a-predictor-serving-cert\""
Sep 11 16:27:01.765691 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.765668    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:27:01.787788 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.787761    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.787904 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.787793    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.787904 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.787815    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.787904 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.787831    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.788035 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.787911    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdrb7\" (UniqueName: \"kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.888943 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.888905    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.888943 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.888938    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889200 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.888972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889200 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:01.889072    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-serving-cert: secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Sep 11 16:27:01.889200 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889102    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889200 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:01.889142    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls podName:6161918e-db69-4222-8340-14c7c05a98d5 nodeName:}" failed. No retries permitted until 2026-09-11 16:27:02.389120623 +0000 UTC m=+578.162360664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls") pod "isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" (UID: "6161918e-db69-4222-8340-14c7c05a98d5") : secret "isvc-sklearn-graph-1-predictor-serving-cert" not found
Sep 11 16:27:01.889200 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889196    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.889499 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.889499 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889281    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdrb7\" (UniqueName: \"kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889499 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889311    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvsr2\" (UniqueName: \"kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.889658 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889536    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889849 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889825    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.889914 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.889837    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.900795 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.900768    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdrb7\" (UniqueName: \"kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:01.966489 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.966393    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:27:01.970272 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.970245    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:01.972782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.972762    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\""
Sep 11 16:27:01.972892 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.972769    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\""
Sep 11 16:27:01.980059 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.980038    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:27:01.990626 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.990587    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.990742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.990637    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.990742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.990682    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvsr2\" (UniqueName: \"kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.992140 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.991714    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:01.994074 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:01.994049    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:02.005903 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.005882    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvsr2\" (UniqueName: \"kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2\") pod \"success-200-isvc-8975a-predictor-67685ddcd7-57fpp\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") " pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:02.067428 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.067392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:02.071270 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.071245    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:27:02.074774 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.074757    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.077768 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.077747    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Sep 11 16:27:02.077768 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.077760    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Sep 11 16:27:02.087189 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.087140    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:27:02.091802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.091525    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.091802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.091574    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.091802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.091620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.091802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.091659    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wn4t\" (UniqueName: \"kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.091802 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.091704    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.192832 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.192793    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193021 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.192841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wn4t\" (UniqueName: \"kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193021 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.192877    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.193021 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.192944    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193028    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46x9j\" (UniqueName: \"kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.193198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.193198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193117    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193144    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.193198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.193424 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193217    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193424 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:02.193349    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found
Sep 11 16:27:02.193424 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193424 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:02.193415    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls podName:3fdf1240-102f-4895-82a8-2f9cc1acb8c5 nodeName:}" failed. No retries permitted until 2026-09-11 16:27:02.693395798 +0000 UTC m=+578.466635830 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-x7bp5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5") : secret "isvc-xgboost-graph-predictor-serving-cert" not found
Sep 11 16:27:02.193691 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193669    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.193909 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.193889    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.204085 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.204058    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wn4t\" (UniqueName: \"kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.294112 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294112 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46x9j\" (UniqueName: \"kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294315 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294116    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294315 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294315 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294169    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294315 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:02.294289    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 11 16:27:02.294547 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:27:02.294353    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls podName:3ca7c6fd-d65b-4007-a311-8ea69cc6593a nodeName:}" failed. No retries permitted until 2026-09-11 16:27:02.794334069 +0000 UTC m=+578.567574096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls") pod "isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found
Sep 11 16:27:02.294616 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294559    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294851 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294828    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.294921 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.294851    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.303573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.303548    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46x9j\" (UniqueName: \"kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.354399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.354363    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:27:02.358059 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:27:02.358031    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod817b4430_0c4f_4d23_af34_12bf6e0fe825.slice/crio-0470fe037adf4481100685e5cde55cfdaafcf9eb3bc2c4e393595ccaf26fcdee WatchSource:0}: Error finding container 0470fe037adf4481100685e5cde55cfdaafcf9eb3bc2c4e393595ccaf26fcdee: Status 404 returned error can't find the container with id 0470fe037adf4481100685e5cde55cfdaafcf9eb3bc2c4e393595ccaf26fcdee
Sep 11 16:27:02.394732 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.394697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:02.397236 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.397213    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:02.567879 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.567783    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:02.697013 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.696969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.699507 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.699489    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-x7bp5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:02.797611 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.797573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.800192 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.800163    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:02.845201 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.844138    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:27:02.848501 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:27:02.847714    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6161918e_db69_4222_8340_14c7c05a98d5.slice/crio-55a70f5966797562fa833ad23dcbe4d5006ceb0d718ce504245a19448743ce52 WatchSource:0}: Error finding container 55a70f5966797562fa833ad23dcbe4d5006ceb0d718ce504245a19448743ce52: Status 404 returned error can't find the container with id 55a70f5966797562fa833ad23dcbe4d5006ceb0d718ce504245a19448743ce52
Sep 11 16:27:02.882021 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:02.881570    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:03.003860 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.003601    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:03.004278 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.004249    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerStarted","Data":"0470fe037adf4481100685e5cde55cfdaafcf9eb3bc2c4e393595ccaf26fcdee"}
Sep 11 16:27:03.013171 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.012209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerStarted","Data":"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"}
Sep 11 16:27:03.013171 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.012287    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerStarted","Data":"55a70f5966797562fa833ad23dcbe4d5006ceb0d718ce504245a19448743ce52"}
Sep 11 16:27:03.185110 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.184898    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:27:03.329311 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:03.329181    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:27:04.023004 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:04.022163    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerStarted","Data":"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"}
Sep 11 16:27:04.023004 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:04.022244    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerStarted","Data":"bc2717680b966c4ab6f049a456c1e4818e1fd04354ee819351bae2e01adefda9"}
Sep 11 16:27:04.027998 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:04.026787    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerStarted","Data":"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"}
Sep 11 16:27:04.027998 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:04.026849    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerStarted","Data":"5a3d27c709f49a72329050efdd2b0df984df29caee20adcecfe490ea777c305f"}
Sep 11 16:27:08.048642 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.048601    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerID="61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764" exitCode=0
Sep 11 16:27:08.049129 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.048684    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerDied","Data":"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"}
Sep 11 16:27:08.050682 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.050655    2579 generic.go:356] "Generic (PLEG): container finished" podID="6161918e-db69-4222-8340-14c7c05a98d5" containerID="f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6" exitCode=0
Sep 11 16:27:08.050770 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.050740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerDied","Data":"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"}
Sep 11 16:27:08.052594 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.052556    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerID="1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2" exitCode=0
Sep 11 16:27:08.052594 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:08.052584    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerDied","Data":"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"}
Sep 11 16:27:19.106370 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:19.106194    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerStarted","Data":"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"}
Sep 11 16:27:19.108856 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:19.108820    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerStarted","Data":"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"}
Sep 11 16:27:21.122402 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:21.122351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerStarted","Data":"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"}
Sep 11 16:27:23.132577 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.132535    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerStarted","Data":"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"}
Sep 11 16:27:23.133085 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.132600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:23.133085 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.132632    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:23.134364 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.134332    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:23.134871 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.134847    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerStarted","Data":"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"}
Sep 11 16:27:23.134967 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.134910    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:23.134967 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.134942    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:23.136218 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.136190    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:27:23.137116 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.137093    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerStarted","Data":"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"}
Sep 11 16:27:23.137232 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.137210    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:23.157232 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.157169    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podStartSLOduration=2.282442106 podStartE2EDuration="22.15715041s" totalImagesPullingTime="19.874708304s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:27:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:27:23.154738583 +0000 UTC m=+598.927978628" watchObservedRunningTime="2026-09-11 16:27:23.15715041 +0000 UTC m=+598.930390441"
Sep 11 16:27:23.175469 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.175398    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podStartSLOduration=3.778245478 podStartE2EDuration="22.175375826s" totalImagesPullingTime="14.158947048s" totalInitContainerRuntime="4.2381833s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:27:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:27:23.174123245 +0000 UTC m=+598.947363278" watchObservedRunningTime="2026-09-11 16:27:23.175375826 +0000 UTC m=+598.948615857"
Sep 11 16:27:23.195652 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:23.195596    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podStartSLOduration=4.756709313 podStartE2EDuration="21.195579035s" totalImagesPullingTime="12.195965656s" totalInitContainerRuntime="4.242904066s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:27:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:27:23.194978461 +0000 UTC m=+598.968218493" watchObservedRunningTime="2026-09-11 16:27:23.195579035 +0000 UTC m=+598.968819066"
Sep 11 16:27:24.141303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:24.141276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:24.141839 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:24.141430    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:24.141839 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:24.141430    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:27:24.142740 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:24.142718    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:27:25.145556 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:25.145512    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:27:29.147059 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:29.147010    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:27:29.147670 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:29.147653    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:27:29.148034 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:29.147993    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:27:29.148168 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:29.148046    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:30.150099 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:30.150071    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:27:30.150600 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:30.150575    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:27:39.147946 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.147907    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:27:39.148350 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.148047    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:39.210102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.210070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerStarted","Data":"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"}
Sep 11 16:27:39.210288 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.210133    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerStarted","Data":"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"}
Sep 11 16:27:39.210427 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.210410    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:39.210583 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.210567    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:39.211731 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.211709    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:27:39.235098 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:39.235047    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podStartSLOduration=3.5676538190000002 podStartE2EDuration="38.235033694s" totalImagesPullingTime="30.472409922s" totalInitContainerRuntime="4.194969953s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:27:39.232600967 +0000 UTC m=+615.005840998" watchObservedRunningTime="2026-09-11 16:27:39.235033694 +0000 UTC m=+615.008273753"
Sep 11 16:27:40.150749 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:40.150715    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:27:40.214004 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:40.213968    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:27:45.218805 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:45.218773    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:27:45.221117 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:45.219261    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:27:49.147595 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:49.147559    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:27:49.148771 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:49.148746    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:50.151056 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:50.151018    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:27:55.219349 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:55.219307    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:27:59.148364 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:59.148324    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:27:59.148775 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:27:59.148324    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:00.150775 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:00.150732    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:28:05.219759 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:05.219716    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:28:09.147806 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:09.147744    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:09.148352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:09.148320    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:28:10.151310 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:10.151265    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:28:15.219316 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:15.219268    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:28:19.148125 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:19.148080    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:19.148847 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:19.148827    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:28:20.150914 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:20.150801    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:28:25.220086 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:25.220039    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:28:29.147792 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:29.147754    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:30.150809 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:30.150764    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:28:35.219959 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:35.219905    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:28:39.147910 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:39.147858    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:39.745880 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:39.745829    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.23:8080: connect: connection refused"
Sep 11 16:28:40.150654 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:40.150615    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.20:8080: connect: connection refused"
Sep 11 16:28:45.219632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.219596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:28:45.855957 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.855914    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:28:45.856243 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.856214    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" containerID="cri-o://a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b" gracePeriod=30
Sep 11 16:28:45.856381 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.856342    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kube-rbac-proxy" containerID="cri-o://2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e" gracePeriod=30
Sep 11 16:28:45.942105 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.942065    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:28:45.944935 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.944635    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:45.950151 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.949828    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9c92a-kube-rbac-proxy-sar-config\""
Sep 11 16:28:45.950433 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.950411    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-9c92a-predictor-serving-cert\""
Sep 11 16:28:45.966032 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:45.965999    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:28:46.098023 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.097987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.098250 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.098045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dmsq\" (UniqueName: \"kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.098250 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.098099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.199398 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.199357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.199664 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.199416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dmsq\" (UniqueName: \"kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.199664 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.199475    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.200123 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.200097    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.202287 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.202258    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.208653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.208626    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dmsq\" (UniqueName: \"kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq\") pod \"success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") " pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.266683 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.266633    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:46.494325 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.494244    2579 generic.go:356] "Generic (PLEG): container finished" podID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerID="2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e" exitCode=2
Sep 11 16:28:46.494325 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.494297    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerDied","Data":"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"}
Sep 11 16:28:46.546578 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.546552    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:28:46.549014 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:28:46.548985    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5826f5e_cffe_45f6_a580_69e2aa5c0c7c.slice/crio-3c025a9ca170c860ec2156bc81e129bb11a01c4460631d9e19b3c88bdd13030c WatchSource:0}: Error finding container 3c025a9ca170c860ec2156bc81e129bb11a01c4460631d9e19b3c88bdd13030c: Status 404 returned error can't find the container with id 3c025a9ca170c860ec2156bc81e129bb11a01c4460631d9e19b3c88bdd13030c
Sep 11 16:28:46.750702 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:46.750619    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:28:47.500109 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.500072    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerStarted","Data":"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"}
Sep 11 16:28:47.500573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.500137    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerStarted","Data":"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"}
Sep 11 16:28:47.500573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.500157    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerStarted","Data":"3c025a9ca170c860ec2156bc81e129bb11a01c4460631d9e19b3c88bdd13030c"}
Sep 11 16:28:47.500573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.500435    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:47.502031 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.501994    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:28:47.520347 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:47.520288    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podStartSLOduration=2.5202688330000003 podStartE2EDuration="2.520268833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:28:47.518580722 +0000 UTC m=+683.291820753" watchObservedRunningTime="2026-09-11 16:28:47.520268833 +0000 UTC m=+683.293508924"
Sep 11 16:28:48.504001 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:48.503963    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:48.504425 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:48.504126    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:28:49.141877 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.141825    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.21:8643/healthz\": dial tcp 10.134.0.21:8643: connect: connection refused"
Sep 11 16:28:49.148188 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.148147    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.21:8080: connect: connection refused"
Sep 11 16:28:49.507942 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.507892    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:28:49.696665 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.696632    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:28:49.746659 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.746564    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:28:49.831217 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.831177    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config\") pod \"817b4430-0c4f-4d23-af34-12bf6e0fe825\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") "
Sep 11 16:28:49.831475 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.831261    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls\") pod \"817b4430-0c4f-4d23-af34-12bf6e0fe825\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") "
Sep 11 16:28:49.831475 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.831313    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvsr2\" (UniqueName: \"kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2\") pod \"817b4430-0c4f-4d23-af34-12bf6e0fe825\" (UID: \"817b4430-0c4f-4d23-af34-12bf6e0fe825\") "
Sep 11 16:28:49.831899 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.831862    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "817b4430-0c4f-4d23-af34-12bf6e0fe825" (UID: "817b4430-0c4f-4d23-af34-12bf6e0fe825"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:28:49.833882 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.833843    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "817b4430-0c4f-4d23-af34-12bf6e0fe825" (UID: "817b4430-0c4f-4d23-af34-12bf6e0fe825"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:28:49.833990 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.833914    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2" (OuterVolumeSpecName: "kube-api-access-dvsr2") pod "817b4430-0c4f-4d23-af34-12bf6e0fe825" (UID: "817b4430-0c4f-4d23-af34-12bf6e0fe825"). InnerVolumeSpecName "kube-api-access-dvsr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:28:49.932381 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.932337    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:28:49.932381 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.932373    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/817b4430-0c4f-4d23-af34-12bf6e0fe825-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:28:49.932381 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:49.932388    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvsr2\" (UniqueName: \"kubernetes.io/projected/817b4430-0c4f-4d23-af34-12bf6e0fe825-kube-api-access-dvsr2\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:28:50.513949 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.513903    2579 generic.go:356] "Generic (PLEG): container finished" podID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerID="a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b" exitCode=0
Sep 11 16:28:50.514415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.513960    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"
Sep 11 16:28:50.514415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.513986    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerDied","Data":"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"}
Sep 11 16:28:50.514415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.514044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp" event={"ID":"817b4430-0c4f-4d23-af34-12bf6e0fe825","Type":"ContainerDied","Data":"0470fe037adf4481100685e5cde55cfdaafcf9eb3bc2c4e393595ccaf26fcdee"}
Sep 11 16:28:50.514415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.514070    2579 scope.go:117] "RemoveContainer" containerID="2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"
Sep 11 16:28:50.523645 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.523623    2579 scope.go:117] "RemoveContainer" containerID="a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"
Sep 11 16:28:50.532399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.532374    2579 scope.go:117] "RemoveContainer" containerID="2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"
Sep 11 16:28:50.532830 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:28:50.532811    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e\": container with ID starting with 2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e not found: ID does not exist" containerID="2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"
Sep 11 16:28:50.532893 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.532844    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e"} err="failed to get container status \"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e\": rpc error: code = NotFound desc = could not find container \"2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e\": container with ID starting with 2753082edb5c4660b8db0673824a58d1349e55c3e8cc763b1ac08da54819d35e not found: ID does not exist"
Sep 11 16:28:50.532893 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.532864    2579 scope.go:117] "RemoveContainer" containerID="a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"
Sep 11 16:28:50.533156 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:28:50.533138    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b\": container with ID starting with a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b not found: ID does not exist" containerID="a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"
Sep 11 16:28:50.533205 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.533162    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b"} err="failed to get container status \"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b\": rpc error: code = NotFound desc = could not find container \"a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b\": container with ID starting with a5c5e027ae90c1fb79b557feea1abc470676edfa282224600f76a30035a6ea9b not found: ID does not exist"
Sep 11 16:28:50.536690 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.536653    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:28:50.540668 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.540638    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8975a-predictor-67685ddcd7-57fpp"]
Sep 11 16:28:50.751193 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:50.751153    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" path="/var/lib/kubelet/pods/817b4430-0c4f-4d23-af34-12bf6e0fe825/volumes"
Sep 11 16:28:54.512978 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:54.512924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:28:54.513712 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:28:54.513679    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:29:04.514254 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:04.514212    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:29:14.513924 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:14.513882    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:29:21.725407 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.725330    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:29:21.725871 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.725647    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container" containerID="cri-o://82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861" gracePeriod=30
Sep 11 16:29:21.725871 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.725786    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kube-rbac-proxy" containerID="cri-o://69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e" gracePeriod=30
Sep 11 16:29:21.770395 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.770365    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:29:21.770641 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.770594    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" containerID="cri-o://432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623" gracePeriod=30
Sep 11 16:29:21.770738 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.770645    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kube-rbac-proxy" containerID="cri-o://7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae" gracePeriod=30
Sep 11 16:29:21.776058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776027    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:29:21.776462 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776426    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kube-rbac-proxy"
Sep 11 16:29:21.776462 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776458    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kube-rbac-proxy"
Sep 11 16:29:21.776620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776481    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container"
Sep 11 16:29:21.776620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776488    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container"
Sep 11 16:29:21.776620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776538    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kube-rbac-proxy"
Sep 11 16:29:21.776620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.776545    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="817b4430-0c4f-4d23-af34-12bf6e0fe825" containerName="kserve-container"
Sep 11 16:29:21.778695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.778677    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.781081 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.781061    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ff919-kube-rbac-proxy-sar-config\""
Sep 11 16:29:21.781165 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.781118    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ff919-predictor-serving-cert\""
Sep 11 16:29:21.789321 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.789298    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:29:21.832269 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.832244    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:29:21.832522 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.832486    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container" containerID="cri-o://d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89" gracePeriod=30
Sep 11 16:29:21.832522 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.832513    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kube-rbac-proxy" containerID="cri-o://97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194" gracePeriod=30
Sep 11 16:29:21.879573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.879547    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jt2b\" (UniqueName: \"kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.879573 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.879584    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.879763 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.879649    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.980311 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.980243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jt2b\" (UniqueName: \"kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.980311 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.980284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.980311 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.980309    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.980911 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.980890    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.983015 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.982985    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:21.990991 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:21.990963    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jt2b\" (UniqueName: \"kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b\") pod \"success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") " pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:22.091493 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.091438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:22.358575 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.358425    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:29:22.361199 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:29:22.361173    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08575857_6e56_4643_98c3_e7947e5c1264.slice/crio-5ea8798cf73bf0bb6f66f95f23e9d32b20246a7269390a5d770579b30881a03c WatchSource:0}: Error finding container 5ea8798cf73bf0bb6f66f95f23e9d32b20246a7269390a5d770579b30881a03c: Status 404 returned error can't find the container with id 5ea8798cf73bf0bb6f66f95f23e9d32b20246a7269390a5d770579b30881a03c
Sep 11 16:29:22.650861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.650832    2579 generic.go:356] "Generic (PLEG): container finished" podID="6161918e-db69-4222-8340-14c7c05a98d5" containerID="69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e" exitCode=2
Sep 11 16:29:22.651043 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.650899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerDied","Data":"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"}
Sep 11 16:29:22.653021 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.652999    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerID="7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae" exitCode=2
Sep 11 16:29:22.653136 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.653068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerDied","Data":"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"}
Sep 11 16:29:22.654753 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.654723    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerStarted","Data":"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"}
Sep 11 16:29:22.654853 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.654774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerStarted","Data":"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"}
Sep 11 16:29:22.654853 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.654802    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerStarted","Data":"5ea8798cf73bf0bb6f66f95f23e9d32b20246a7269390a5d770579b30881a03c"}
Sep 11 16:29:22.654853 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.654833    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:22.654992 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.654858    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:22.656355 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.656329    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:29:22.656893 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.656875    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerID="97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194" exitCode=2
Sep 11 16:29:22.656973 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.656913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerDied","Data":"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"}
Sep 11 16:29:22.674442 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:22.673117    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podStartSLOduration=1.673096725 podStartE2EDuration="1.673096725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:29:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:29:22.671783896 +0000 UTC m=+718.445023928" watchObservedRunningTime="2026-09-11 16:29:22.673096725 +0000 UTC m=+718.446336763"
Sep 11 16:29:23.663533 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:23.663492    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:29:24.141913 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:24.141880    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.23:8643/healthz\": dial tcp 10.134.0.23:8643: connect: connection refused"
Sep 11 16:29:24.514217 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:24.514130    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.24:8080: connect: connection refused"
Sep 11 16:29:25.146589 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.146550    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.20:8643/healthz\": dial tcp 10.134.0.20:8643: connect: connection refused"
Sep 11 16:29:25.214953 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.214910    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.22:8643/healthz\": dial tcp 10.134.0.22:8643: connect: connection refused"
Sep 11 16:29:25.219212 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.219185    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.22:8080: connect: connection refused"
Sep 11 16:29:25.511998 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.511975    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:29:25.611602 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.611574    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") pod \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") "
Sep 11 16:29:25.611787 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.611628    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert\") pod \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") "
Sep 11 16:29:25.611787 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.611689    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config\") pod \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") "
Sep 11 16:29:25.611787 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.611719    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location\") pod \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") "
Sep 11 16:29:25.611787 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.611749    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wn4t\" (UniqueName: \"kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t\") pod \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\" (UID: \"3fdf1240-102f-4895-82a8-2f9cc1acb8c5\") "
Sep 11 16:29:25.612052 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.612017    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3fdf1240-102f-4895-82a8-2f9cc1acb8c5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:25.612052 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.612039    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3fdf1240-102f-4895-82a8-2f9cc1acb8c5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:25.612225 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.612069    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3fdf1240-102f-4895-82a8-2f9cc1acb8c5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:29:25.613865 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.613842    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3fdf1240-102f-4895-82a8-2f9cc1acb8c5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:29:25.613950 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.613922    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t" (OuterVolumeSpecName: "kube-api-access-2wn4t") pod "3fdf1240-102f-4895-82a8-2f9cc1acb8c5" (UID: "3fdf1240-102f-4895-82a8-2f9cc1acb8c5"). InnerVolumeSpecName "kube-api-access-2wn4t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:29:25.675604 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.675571    2579 generic.go:356] "Generic (PLEG): container finished" podID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerID="432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623" exitCode=0
Sep 11 16:29:25.675776 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.675632    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerDied","Data":"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"}
Sep 11 16:29:25.675776 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.675649    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"
Sep 11 16:29:25.675776 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.675708    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5" event={"ID":"3fdf1240-102f-4895-82a8-2f9cc1acb8c5","Type":"ContainerDied","Data":"5a3d27c709f49a72329050efdd2b0df984df29caee20adcecfe490ea777c305f"}
Sep 11 16:29:25.675776 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.675738    2579 scope.go:117] "RemoveContainer" containerID="7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"
Sep 11 16:29:25.684879 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.684738    2579 scope.go:117] "RemoveContainer" containerID="432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"
Sep 11 16:29:25.692172 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.692155    2579 scope.go:117] "RemoveContainer" containerID="1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"
Sep 11 16:29:25.697291 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.697268    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:29:25.699323 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.699302    2579 scope.go:117] "RemoveContainer" containerID="7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"
Sep 11 16:29:25.699717 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:25.699674    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae\": container with ID starting with 7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae not found: ID does not exist" containerID="7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"
Sep 11 16:29:25.699807 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.699728    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae"} err="failed to get container status \"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae\": rpc error: code = NotFound desc = could not find container \"7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae\": container with ID starting with 7d916de8ee3285c1b29c001faa8bf34f76c9ba44536c4520d9f1265bb33cd0ae not found: ID does not exist"
Sep 11 16:29:25.699807 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.699751    2579 scope.go:117] "RemoveContainer" containerID="432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"
Sep 11 16:29:25.700112 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:25.700093    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623\": container with ID starting with 432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623 not found: ID does not exist" containerID="432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"
Sep 11 16:29:25.700196 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.700115    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623"} err="failed to get container status \"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623\": rpc error: code = NotFound desc = could not find container \"432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623\": container with ID starting with 432aa3069e5732b11d98347c89952db31766ddf5119416118740c0f4431c6623 not found: ID does not exist"
Sep 11 16:29:25.700196 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.700130    2579 scope.go:117] "RemoveContainer" containerID="1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"
Sep 11 16:29:25.700407 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:25.700390    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2\": container with ID starting with 1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2 not found: ID does not exist" containerID="1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"
Sep 11 16:29:25.700505 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.700416    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2"} err="failed to get container status \"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2\": rpc error: code = NotFound desc = could not find container \"1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2\": container with ID starting with 1642c32ea259c613d6ed1b115cac45c61cac01502da76a5e1146c12f951c5dd2 not found: ID does not exist"
Sep 11 16:29:25.701264 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.701247    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-x7bp5"]
Sep 11 16:29:25.712792 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.712774    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:25.712862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.712794    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kserve-provision-location\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:25.712862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.712803    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wn4t\" (UniqueName: \"kubernetes.io/projected/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-kube-api-access-2wn4t\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:25.712862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.712812    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:25.712862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:25.712821    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3fdf1240-102f-4895-82a8-2f9cc1acb8c5-cabundle-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.582146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.582122    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:29:26.619494 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619471    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location\") pod \"6161918e-db69-4222-8340-14c7c05a98d5\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") "
Sep 11 16:29:26.619643 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619508    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config\") pod \"6161918e-db69-4222-8340-14c7c05a98d5\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") "
Sep 11 16:29:26.619643 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619602    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") pod \"6161918e-db69-4222-8340-14c7c05a98d5\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") "
Sep 11 16:29:26.619643 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert\") pod \"6161918e-db69-4222-8340-14c7c05a98d5\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") "
Sep 11 16:29:26.619803 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619661    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdrb7\" (UniqueName: \"kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7\") pod \"6161918e-db69-4222-8340-14c7c05a98d5\" (UID: \"6161918e-db69-4222-8340-14c7c05a98d5\") "
Sep 11 16:29:26.619862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619818    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "6161918e-db69-4222-8340-14c7c05a98d5" (UID: "6161918e-db69-4222-8340-14c7c05a98d5"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:29:26.619955 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619936    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6161918e-db69-4222-8340-14c7c05a98d5-kserve-provision-location\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.620007 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.619947    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "6161918e-db69-4222-8340-14c7c05a98d5" (UID: "6161918e-db69-4222-8340-14c7c05a98d5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:26.620043 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.620018    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "6161918e-db69-4222-8340-14c7c05a98d5" (UID: "6161918e-db69-4222-8340-14c7c05a98d5"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:26.621813 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.621790    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7" (OuterVolumeSpecName: "kube-api-access-rdrb7") pod "6161918e-db69-4222-8340-14c7c05a98d5" (UID: "6161918e-db69-4222-8340-14c7c05a98d5"). InnerVolumeSpecName "kube-api-access-rdrb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:29:26.621912 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.621859    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6161918e-db69-4222-8340-14c7c05a98d5" (UID: "6161918e-db69-4222-8340-14c7c05a98d5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:29:26.681200 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.681166    2579 generic.go:356] "Generic (PLEG): container finished" podID="6161918e-db69-4222-8340-14c7c05a98d5" containerID="82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861" exitCode=0
Sep 11 16:29:26.681337 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.681227    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"
Sep 11 16:29:26.681337 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.681245    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerDied","Data":"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"}
Sep 11 16:29:26.681337 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.681298    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf" event={"ID":"6161918e-db69-4222-8340-14c7c05a98d5","Type":"ContainerDied","Data":"55a70f5966797562fa833ad23dcbe4d5006ceb0d718ce504245a19448743ce52"}
Sep 11 16:29:26.681337 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.681325    2579 scope.go:117] "RemoveContainer" containerID="69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"
Sep 11 16:29:26.689744 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.689518    2579 scope.go:117] "RemoveContainer" containerID="82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"
Sep 11 16:29:26.696630 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.696612    2579 scope.go:117] "RemoveContainer" containerID="f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"
Sep 11 16:29:26.702681 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.702661    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:29:26.704164 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.704145    2579 scope.go:117] "RemoveContainer" containerID="69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"
Sep 11 16:29:26.704399 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:26.704377    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e\": container with ID starting with 69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e not found: ID does not exist" containerID="69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"
Sep 11 16:29:26.704526 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.704407    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e"} err="failed to get container status \"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e\": rpc error: code = NotFound desc = could not find container \"69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e\": container with ID starting with 69e9c931832bcebaeaf5f6739fc360cfee6f97135f2652492fe4f274910b082e not found: ID does not exist"
Sep 11 16:29:26.704526 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.704422    2579 scope.go:117] "RemoveContainer" containerID="82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"
Sep 11 16:29:26.704768 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:26.704716    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861\": container with ID starting with 82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861 not found: ID does not exist" containerID="82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"
Sep 11 16:29:26.704861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.704778    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861"} err="failed to get container status \"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861\": rpc error: code = NotFound desc = could not find container \"82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861\": container with ID starting with 82e2fd6bc55913236c636fc1e4c570845cdda20a4c1798e2f45e5024a7405861 not found: ID does not exist"
Sep 11 16:29:26.704861 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.704826    2579 scope.go:117] "RemoveContainer" containerID="f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"
Sep 11 16:29:26.705218 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:26.705198    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6\": container with ID starting with f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6 not found: ID does not exist" containerID="f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"
Sep 11 16:29:26.705302 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.705221    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6"} err="failed to get container status \"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6\": rpc error: code = NotFound desc = could not find container \"f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6\": container with ID starting with f9eba29b2b908d86a93955a42e1b6ab72c5808743b71cedf887702bf8fbdf9d6 not found: ID does not exist"
Sep 11 16:29:26.706819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.706798    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-5dcd4747bf-pxvpf"]
Sep 11 16:29:26.720696 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.720680    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6161918e-db69-4222-8340-14c7c05a98d5-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.720761 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.720698    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-cabundle-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.720761 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.720707    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdrb7\" (UniqueName: \"kubernetes.io/projected/6161918e-db69-4222-8340-14c7c05a98d5-kube-api-access-rdrb7\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.720761 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.720717    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/6161918e-db69-4222-8340-14c7c05a98d5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:26.749105 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.749084    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" path="/var/lib/kubelet/pods/3fdf1240-102f-4895-82a8-2f9cc1acb8c5/volumes"
Sep 11 16:29:26.749559 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:26.749546    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6161918e-db69-4222-8340-14c7c05a98d5" path="/var/lib/kubelet/pods/6161918e-db69-4222-8340-14c7c05a98d5/volumes"
Sep 11 16:29:27.567553 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.567522    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:29:27.625118 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625042    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config\") pod \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") "
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625157    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert\") pod \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") "
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625208    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") pod \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") "
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location\") pod \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") "
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46x9j\" (UniqueName: \"kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j\") pod \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\" (UID: \"3ca7c6fd-d65b-4007-a311-8ea69cc6593a\") "
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625410    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "3ca7c6fd-d65b-4007-a311-8ea69cc6593a" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:27.625550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625499    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "3ca7c6fd-d65b-4007-a311-8ea69cc6593a" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:29:27.625859 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625651    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:27.625859 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.625650    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "3ca7c6fd-d65b-4007-a311-8ea69cc6593a" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 16:29:27.627407 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.627388    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3ca7c6fd-d65b-4007-a311-8ea69cc6593a" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:29:27.627487 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.627422    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j" (OuterVolumeSpecName: "kube-api-access-46x9j") pod "3ca7c6fd-d65b-4007-a311-8ea69cc6593a" (UID: "3ca7c6fd-d65b-4007-a311-8ea69cc6593a"). InnerVolumeSpecName "kube-api-access-46x9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:29:27.687905 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.687873    2579 generic.go:356] "Generic (PLEG): container finished" podID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerID="d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89" exitCode=0
Sep 11 16:29:27.688058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.687929    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"
Sep 11 16:29:27.688058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.687960    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerDied","Data":"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"}
Sep 11 16:29:27.688058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.688025    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg" event={"ID":"3ca7c6fd-d65b-4007-a311-8ea69cc6593a","Type":"ContainerDied","Data":"bc2717680b966c4ab6f049a456c1e4818e1fd04354ee819351bae2e01adefda9"}
Sep 11 16:29:27.688058 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.688052    2579 scope.go:117] "RemoveContainer" containerID="97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"
Sep 11 16:29:27.696803 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.696525    2579 scope.go:117] "RemoveContainer" containerID="d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"
Sep 11 16:29:27.703701 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.703685    2579 scope.go:117] "RemoveContainer" containerID="61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"
Sep 11 16:29:27.709292 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.709268    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:29:27.711521 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.711509    2579 scope.go:117] "RemoveContainer" containerID="97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"
Sep 11 16:29:27.711798 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:27.711778    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194\": container with ID starting with 97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194 not found: ID does not exist" containerID="97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"
Sep 11 16:29:27.711842 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.711806    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194"} err="failed to get container status \"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194\": rpc error: code = NotFound desc = could not find container \"97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194\": container with ID starting with 97355ed63969f668783fef8493af223a5938376b8ab0bf21738b11f19c351194 not found: ID does not exist"
Sep 11 16:29:27.711842 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.711823    2579 scope.go:117] "RemoveContainer" containerID="d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"
Sep 11 16:29:27.712090 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:27.712072    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89\": container with ID starting with d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89 not found: ID does not exist" containerID="d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"
Sep 11 16:29:27.712141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.712095    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89"} err="failed to get container status \"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89\": rpc error: code = NotFound desc = could not find container \"d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89\": container with ID starting with d35f9555647885a6b7cb7c87ab558acb4e68e0a7a860a03bfeaae0abcc716c89 not found: ID does not exist"
Sep 11 16:29:27.712141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.712108    2579 scope.go:117] "RemoveContainer" containerID="61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"
Sep 11 16:29:27.712325 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:29:27.712309    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764\": container with ID starting with 61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764 not found: ID does not exist" containerID="61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"
Sep 11 16:29:27.712365 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.712330    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764"} err="failed to get container status \"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764\": rpc error: code = NotFound desc = could not find container \"61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764\": container with ID starting with 61b06abdaf571c034aa659568a7c2f77358dbd6dbf6d21f028983d34ce070764 not found: ID does not exist"
Sep 11 16:29:27.717070 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.717046    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-76586f68dc-g5gwg"]
Sep 11 16:29:27.726009 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.725986    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:27.726009 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.726005    2579 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kserve-provision-location\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:27.726119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.726016    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46x9j\" (UniqueName: \"kubernetes.io/projected/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-kube-api-access-46x9j\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:27.726119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:27.726025    2579 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/3ca7c6fd-d65b-4007-a311-8ea69cc6593a-cabundle-cert\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:29:28.668469 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:28.668423    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:29:28.668952 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:28.668889    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:29:28.749565 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:28.749534    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" path="/var/lib/kubelet/pods/3ca7c6fd-d65b-4007-a311-8ea69cc6593a/volumes"
Sep 11 16:29:34.514632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:34.514600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:29:38.668930 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:38.668877    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:29:48.669011 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:48.668942    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:29:58.668906 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:29:58.668861    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:30:08.670463 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:30:08.670411    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:38:00.767419 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.767387    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:38:00.768003 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.767629    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container" containerID="cri-o://149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995" gracePeriod=30
Sep 11 16:38:00.768003 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.767681    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kube-rbac-proxy" containerID="cri-o://2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785" gracePeriod=30
Sep 11 16:38:00.868702 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.868667    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:38:00.869103 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869085    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869108    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869119    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container"
Sep 11 16:38:00.869146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869127    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container"
Sep 11 16:38:00.869146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869143    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869153    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869165    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869174    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869185    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869195    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="storage-initializer"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869205    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869213    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869224    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869232    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869244    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869252    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869265 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869263    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869272    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869341    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kserve-container"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869351    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869360    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kserve-container"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869368    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6161918e-db69-4222-8340-14c7c05a98d5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869376    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fdf1240-102f-4895-82a8-2f9cc1acb8c5" containerName="kube-rbac-proxy"
Sep 11 16:38:00.869653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.869385    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ca7c6fd-d65b-4007-a311-8ea69cc6593a" containerName="kserve-container"
Sep 11 16:38:00.872695 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.872672    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:00.878151 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.878041    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a9581-predictor-serving-cert\""
Sep 11 16:38:00.878151 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.878045    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-a9581-kube-rbac-proxy-sar-config\""
Sep 11 16:38:00.900499 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:00.900476    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:38:01.018154 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.018073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvv4w\" (UniqueName: \"kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.018154 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.018124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.018359 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.018232    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.119546 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.119508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.119710 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.119570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvv4w\" (UniqueName: \"kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.119710 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.119604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.119811 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:01.119746    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-a9581-predictor-serving-cert: secret "success-200-isvc-a9581-predictor-serving-cert" not found
Sep 11 16:38:01.119811 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:01.119809    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls podName:495ada4f-7c74-4cc7-9c2f-12d7daba09f7 nodeName:}" failed. No retries permitted until 2026-09-11 16:38:01.619787693 +0000 UTC m=+1237.393027715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls") pod "success-200-isvc-a9581-predictor-5656556755-688q4" (UID: "495ada4f-7c74-4cc7-9c2f-12d7daba09f7") : secret "success-200-isvc-a9581-predictor-serving-cert" not found
Sep 11 16:38:01.120143 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.120123    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.128740 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.128720    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvv4w\" (UniqueName: \"kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.559717 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.559680    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerID="2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785" exitCode=2
Sep 11 16:38:01.559899 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.559725    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerDied","Data":"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"}
Sep 11 16:38:01.623219 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.623187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.625709 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.625687    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") pod \"success-200-isvc-a9581-predictor-5656556755-688q4\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") " pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:01.784358 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:01.784323    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:02.052936 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.052906    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:38:02.055049 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:38:02.055028    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod495ada4f_7c74_4cc7_9c2f_12d7daba09f7.slice/crio-1724ff4f36d732d70ac40c589c9ef831231a76e5d7a3b62df79c6e1097650c48 WatchSource:0}: Error finding container 1724ff4f36d732d70ac40c589c9ef831231a76e5d7a3b62df79c6e1097650c48: Status 404 returned error can't find the container with id 1724ff4f36d732d70ac40c589c9ef831231a76e5d7a3b62df79c6e1097650c48
Sep 11 16:38:02.565198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.565164    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerStarted","Data":"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"}
Sep 11 16:38:02.565399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.565226    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerStarted","Data":"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"}
Sep 11 16:38:02.565399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.565256    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerStarted","Data":"1724ff4f36d732d70ac40c589c9ef831231a76e5d7a3b62df79c6e1097650c48"}
Sep 11 16:38:02.565399 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.565287    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:02.582287 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:02.582237    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podStartSLOduration=2.582223721 podStartE2EDuration="2.582223721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:38:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:38:02.581076971 +0000 UTC m=+1238.354317000" watchObservedRunningTime="2026-09-11 16:38:02.582223721 +0000 UTC m=+1238.355463750"
Sep 11 16:38:03.569504 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.569471    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:03.570775 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.570752    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:03.707703 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.707678    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:38:03.839718 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.839664    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dmsq\" (UniqueName: \"kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq\") pod \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") "
Sep 11 16:38:03.839828 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.839726    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config\") pod \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") "
Sep 11 16:38:03.839828 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.839746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls\") pod \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\" (UID: \"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c\") "
Sep 11 16:38:03.840005 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.839985    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" (UID: "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:38:03.841847 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.841819    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" (UID: "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:38:03.841926 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.841886    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq" (OuterVolumeSpecName: "kube-api-access-6dmsq") pod "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" (UID: "b5826f5e-cffe-45f6-a580-69e2aa5c0c7c"). InnerVolumeSpecName "kube-api-access-6dmsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:38:03.940413 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.940390    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:03.940413 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.940408    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:03.940547 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:03.940419    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dmsq\" (UniqueName: \"kubernetes.io/projected/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c-kube-api-access-6dmsq\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:04.574174 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574138    2579 generic.go:356] "Generic (PLEG): container finished" podID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerID="149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995" exitCode=0
Sep 11 16:38:04.574650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574201    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"
Sep 11 16:38:04.574650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574202    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerDied","Data":"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"}
Sep 11 16:38:04.574650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574275    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg" event={"ID":"b5826f5e-cffe-45f6-a580-69e2aa5c0c7c","Type":"ContainerDied","Data":"3c025a9ca170c860ec2156bc81e129bb11a01c4460631d9e19b3c88bdd13030c"}
Sep 11 16:38:04.574650 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574317    2579 scope.go:117] "RemoveContainer" containerID="2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"
Sep 11 16:38:04.574866 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.574789    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:04.582954 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.582800    2579 scope.go:117] "RemoveContainer" containerID="149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"
Sep 11 16:38:04.589950 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.589935    2579 scope.go:117] "RemoveContainer" containerID="2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"
Sep 11 16:38:04.590206 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:04.590189    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785\": container with ID starting with 2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785 not found: ID does not exist" containerID="2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"
Sep 11 16:38:04.590251 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.590213    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785"} err="failed to get container status \"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785\": rpc error: code = NotFound desc = could not find container \"2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785\": container with ID starting with 2f162b5cb2a57b5d19a0a80c63a08d2fa79f26f86d4c3b3a41f158faa9fd7785 not found: ID does not exist"
Sep 11 16:38:04.590251 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.590229    2579 scope.go:117] "RemoveContainer" containerID="149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"
Sep 11 16:38:04.590497 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:04.590480    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995\": container with ID starting with 149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995 not found: ID does not exist" containerID="149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"
Sep 11 16:38:04.590560 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.590501    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995"} err="failed to get container status \"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995\": rpc error: code = NotFound desc = could not find container \"149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995\": container with ID starting with 149c7f7bcf48508479e2b49a4fc672ab2952d8e09205680c45f0e4bd3eb8c995 not found: ID does not exist"
Sep 11 16:38:04.595557 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.595528    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:38:04.599263 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.599243    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-9c92a-predictor-5b64d8b4b7-rk5wg"]
Sep 11 16:38:04.749130 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:04.749106    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" path="/var/lib/kubelet/pods/b5826f5e-cffe-45f6-a580-69e2aa5c0c7c/volumes"
Sep 11 16:38:09.578863 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:09.578835    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:09.579284 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:09.579260    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:19.579571 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:19.579494    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:29.580037 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:29.579994    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:36.618862 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.618825    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:38:36.619352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.619079    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" containerID="cri-o://2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563" gracePeriod=30
Sep 11 16:38:36.619352 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.619130    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kube-rbac-proxy" containerID="cri-o://4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c" gracePeriod=30
Sep 11 16:38:36.716105 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716074    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:38:36.716501 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716487    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container"
Sep 11 16:38:36.716549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716504    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container"
Sep 11 16:38:36.716549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716533    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kube-rbac-proxy"
Sep 11 16:38:36.716549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kube-rbac-proxy"
Sep 11 16:38:36.716640 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716597    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kserve-container"
Sep 11 16:38:36.716640 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.716608    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5826f5e-cffe-45f6-a580-69e2aa5c0c7c" containerName="kube-rbac-proxy"
Sep 11 16:38:36.721198 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.721176    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.723886 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.723852    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e5408-kube-rbac-proxy-sar-config\""
Sep 11 16:38:36.723886 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.723864    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e5408-predictor-serving-cert\""
Sep 11 16:38:36.730144 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.730122    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:38:36.875946 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.875864    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn469\" (UniqueName: \"kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.875946 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.875924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.876126 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.875966    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.976843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.976811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn469\" (UniqueName: \"kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.976843 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.976855    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.977099 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.976878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.977099 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:36.976980    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-e5408-predictor-serving-cert: secret "success-200-isvc-e5408-predictor-serving-cert" not found
Sep 11 16:38:36.977099 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:36.977049    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls podName:d612dee1-fcd2-405b-bbae-a5ef4927efc2 nodeName:}" failed. No retries permitted until 2026-09-11 16:38:37.477030097 +0000 UTC m=+1273.250270122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls") pod "success-200-isvc-e5408-predictor-765956d4cf-svlxt" (UID: "d612dee1-fcd2-405b-bbae-a5ef4927efc2") : secret "success-200-isvc-e5408-predictor-serving-cert" not found
Sep 11 16:38:36.977631 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.977609    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:36.986098 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:36.986068    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn469\" (UniqueName: \"kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:37.480973 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.480936    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:37.483396 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.483365    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") pod \"success-200-isvc-e5408-predictor-765956d4cf-svlxt\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") " pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:37.633809 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.633777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:37.692531 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.692493    2579 generic.go:356] "Generic (PLEG): container finished" podID="08575857-6e56-4643-98c3-e7947e5c1264" containerID="4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c" exitCode=2
Sep 11 16:38:37.692663 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.692585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerDied","Data":"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"}
Sep 11 16:38:37.902161 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:37.902134    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:38:37.905200 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:38:37.905173    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd612dee1_fcd2_405b_bbae_a5ef4927efc2.slice/crio-6d8284841a45ec135ca22d1987e541d93182ebee7de3dbc1cc1d5c17fdfc1812 WatchSource:0}: Error finding container 6d8284841a45ec135ca22d1987e541d93182ebee7de3dbc1cc1d5c17fdfc1812: Status 404 returned error can't find the container with id 6d8284841a45ec135ca22d1987e541d93182ebee7de3dbc1cc1d5c17fdfc1812
Sep 11 16:38:38.664268 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.664221    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.25:8643/healthz\": dial tcp 10.134.0.25:8643: connect: connection refused"
Sep 11 16:38:38.669642 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.669603    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.25:8080: connect: connection refused"
Sep 11 16:38:38.698636 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.698601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerStarted","Data":"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"}
Sep 11 16:38:38.698794 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.698695    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerStarted","Data":"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"}
Sep 11 16:38:38.698794 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.698730    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:38.698794 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.698760    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerStarted","Data":"6d8284841a45ec135ca22d1987e541d93182ebee7de3dbc1cc1d5c17fdfc1812"}
Sep 11 16:38:38.698794 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.698785    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:38.700125 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.700101    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:38:38.718324 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:38.718286    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podStartSLOduration=2.7182747430000003 podStartE2EDuration="2.718274743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:38:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:38:38.717157105 +0000 UTC m=+1274.490397134" watchObservedRunningTime="2026-09-11 16:38:38.718274743 +0000 UTC m=+1274.491514773"
Sep 11 16:38:39.547599 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.547571    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:38:39.579799 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.579765    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.26:8080: connect: connection refused"
Sep 11 16:38:39.595422 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.595400    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jt2b\" (UniqueName: \"kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b\") pod \"08575857-6e56-4643-98c3-e7947e5c1264\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") "
Sep 11 16:38:39.595561 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.595435    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config\") pod \"08575857-6e56-4643-98c3-e7947e5c1264\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") "
Sep 11 16:38:39.595561 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.595476    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls\") pod \"08575857-6e56-4643-98c3-e7947e5c1264\" (UID: \"08575857-6e56-4643-98c3-e7947e5c1264\") "
Sep 11 16:38:39.595817 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.595791    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "08575857-6e56-4643-98c3-e7947e5c1264" (UID: "08575857-6e56-4643-98c3-e7947e5c1264"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:38:39.597603 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.597580    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "08575857-6e56-4643-98c3-e7947e5c1264" (UID: "08575857-6e56-4643-98c3-e7947e5c1264"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:38:39.597661 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.597615    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b" (OuterVolumeSpecName: "kube-api-access-7jt2b") pod "08575857-6e56-4643-98c3-e7947e5c1264" (UID: "08575857-6e56-4643-98c3-e7947e5c1264"). InnerVolumeSpecName "kube-api-access-7jt2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:38:39.696205 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.696146    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jt2b\" (UniqueName: \"kubernetes.io/projected/08575857-6e56-4643-98c3-e7947e5c1264-kube-api-access-7jt2b\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:39.696205 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.696170    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/08575857-6e56-4643-98c3-e7947e5c1264-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:39.696205 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.696179    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/08575857-6e56-4643-98c3-e7947e5c1264-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:38:39.703588 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.703565    2579 generic.go:356] "Generic (PLEG): container finished" podID="08575857-6e56-4643-98c3-e7947e5c1264" containerID="2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563" exitCode=0
Sep 11 16:38:39.703694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.703611    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"
Sep 11 16:38:39.703694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.703651    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerDied","Data":"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"}
Sep 11 16:38:39.703782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.703712    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn" event={"ID":"08575857-6e56-4643-98c3-e7947e5c1264","Type":"ContainerDied","Data":"5ea8798cf73bf0bb6f66f95f23e9d32b20246a7269390a5d770579b30881a03c"}
Sep 11 16:38:39.703782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.703744    2579 scope.go:117] "RemoveContainer" containerID="4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"
Sep 11 16:38:39.704156 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.704122    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:38:39.712059 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.712033    2579 scope.go:117] "RemoveContainer" containerID="2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"
Sep 11 16:38:39.718981 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.718966    2579 scope.go:117] "RemoveContainer" containerID="4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"
Sep 11 16:38:39.719250 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:39.719234    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c\": container with ID starting with 4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c not found: ID does not exist" containerID="4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"
Sep 11 16:38:39.719320 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.719263    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c"} err="failed to get container status \"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c\": rpc error: code = NotFound desc = could not find container \"4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c\": container with ID starting with 4b8e4e72f9c4394d588368aef919d224f31f8280b58e78552173b2b8e1e0829c not found: ID does not exist"
Sep 11 16:38:39.719320 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.719284    2579 scope.go:117] "RemoveContainer" containerID="2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"
Sep 11 16:38:39.719555 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:38:39.719539    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563\": container with ID starting with 2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563 not found: ID does not exist" containerID="2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"
Sep 11 16:38:39.719592 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.719561    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563"} err="failed to get container status \"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563\": rpc error: code = NotFound desc = could not find container \"2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563\": container with ID starting with 2298f29be27b6569026f6c8933919a8b4e0c692c6b800c916eadea727ce02563 not found: ID does not exist"
Sep 11 16:38:39.726002 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.725984    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:38:39.731017 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:39.730998    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ff919-predictor-6bbdbd4fc7-7vgxn"]
Sep 11 16:38:40.749018 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:40.748981    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08575857-6e56-4643-98c3-e7947e5c1264" path="/var/lib/kubelet/pods/08575857-6e56-4643-98c3-e7947e5c1264/volumes"
Sep 11 16:38:44.708766 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:44.708736    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:38:44.709292 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:44.709265    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:38:49.580633 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:49.580596    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:38:54.709525 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:38:54.709488    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:39:04.710112 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:04.710073    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:39:11.046185 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.046107    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:39:11.046688 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.046371    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container" containerID="cri-o://b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00" gracePeriod=30
Sep 11 16:39:11.046688 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.046565    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kube-rbac-proxy" containerID="cri-o://4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039" gracePeriod=30
Sep 11 16:39:11.126985 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.126957    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:39:11.127219 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127207    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kube-rbac-proxy"
Sep 11 16:39:11.127219 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127221    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kube-rbac-proxy"
Sep 11 16:39:11.127303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127231    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container"
Sep 11 16:39:11.127303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127236    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container"
Sep 11 16:39:11.127373 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127305    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kserve-container"
Sep 11 16:39:11.127373 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.127311    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="08575857-6e56-4643-98c3-e7947e5c1264" containerName="kube-rbac-proxy"
Sep 11 16:39:11.130427 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.130412    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.132812 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.132789    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1ab9e-predictor-serving-cert\""
Sep 11 16:39:11.132941 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.132896    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1ab9e-kube-rbac-proxy-sar-config\""
Sep 11 16:39:11.140896 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.140875    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:39:11.211157 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.211125    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.211303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.211165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.211303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.211276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj5hz\" (UniqueName: \"kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.312589 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.312509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj5hz\" (UniqueName: \"kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.312589 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.312566    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.312762 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.312595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.312762 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:11.312748    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-serving-cert: secret "success-200-isvc-1ab9e-predictor-serving-cert" not found
Sep 11 16:39:11.312858 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:11.312820    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls podName:96ef3781-b318-49f5-a13d-f7cc49f05109 nodeName:}" failed. No retries permitted until 2026-09-11 16:39:11.812800125 +0000 UTC m=+1307.586040159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls") pod "success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" (UID: "96ef3781-b318-49f5-a13d-f7cc49f05109") : secret "success-200-isvc-1ab9e-predictor-serving-cert" not found
Sep 11 16:39:11.313256 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.313240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.321601 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.321580    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj5hz\" (UniqueName: \"kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.816303 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.816264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:11.817751 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.817722    2579 generic.go:356] "Generic (PLEG): container finished" podID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerID="4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039" exitCode=2
Sep 11 16:39:11.817872 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.817792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerDied","Data":"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"}
Sep 11 16:39:11.818897 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:11.818873    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") pod \"success-200-isvc-1ab9e-predictor-595c484fbc-shn4m\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") " pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:12.039937 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.039904    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:12.308797 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.308769    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:39:12.311190 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:39:12.311158    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ef3781_b318_49f5_a13d_f7cc49f05109.slice/crio-27bdb565421cb3d63c5b530d0ff6984124ac6f16a6802b4149bd1b420ff2ea15 WatchSource:0}: Error finding container 27bdb565421cb3d63c5b530d0ff6984124ac6f16a6802b4149bd1b420ff2ea15: Status 404 returned error can't find the container with id 27bdb565421cb3d63c5b530d0ff6984124ac6f16a6802b4149bd1b420ff2ea15
Sep 11 16:39:12.822909 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.822875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerStarted","Data":"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"}
Sep 11 16:39:12.823101 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.822934    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerStarted","Data":"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"}
Sep 11 16:39:12.823101 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.822954    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerStarted","Data":"27bdb565421cb3d63c5b530d0ff6984124ac6f16a6802b4149bd1b420ff2ea15"}
Sep 11 16:39:12.823101 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.823020    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:12.841620 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:12.841579    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podStartSLOduration=1.841565359 podStartE2EDuration="1.841565359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:12.839580234 +0000 UTC m=+1308.612820264" watchObservedRunningTime="2026-09-11 16:39:12.841565359 +0000 UTC m=+1308.614805455"
Sep 11 16:39:13.827517 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:13.827485    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:13.828777 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:13.828748    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:14.078755 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.078705    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:39:14.132987 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.132961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") pod \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") "
Sep 11 16:39:14.133094 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.133053    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvv4w\" (UniqueName: \"kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w\") pod \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") "
Sep 11 16:39:14.133094 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.133082    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config\") pod \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\" (UID: \"495ada4f-7c74-4cc7-9c2f-12d7daba09f7\") "
Sep 11 16:39:14.133437 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.133412    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "495ada4f-7c74-4cc7-9c2f-12d7daba09f7" (UID: "495ada4f-7c74-4cc7-9c2f-12d7daba09f7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:39:14.135007 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.134980    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "495ada4f-7c74-4cc7-9c2f-12d7daba09f7" (UID: "495ada4f-7c74-4cc7-9c2f-12d7daba09f7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:39:14.135108 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.135072    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w" (OuterVolumeSpecName: "kube-api-access-nvv4w") pod "495ada4f-7c74-4cc7-9c2f-12d7daba09f7" (UID: "495ada4f-7c74-4cc7-9c2f-12d7daba09f7"). InnerVolumeSpecName "kube-api-access-nvv4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:39:14.234164 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.234141    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvv4w\" (UniqueName: \"kubernetes.io/projected/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-api-access-nvv4w\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:14.234164 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.234163    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:14.234319 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.234172    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/495ada4f-7c74-4cc7-9c2f-12d7daba09f7-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:14.709174 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.709134    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused"
Sep 11 16:39:14.832025 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.831994    2579 generic.go:356] "Generic (PLEG): container finished" podID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerID="b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00" exitCode=0
Sep 11 16:39:14.832390 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.832050    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"
Sep 11 16:39:14.832390 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.832077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerDied","Data":"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"}
Sep 11 16:39:14.832390 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.832154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4" event={"ID":"495ada4f-7c74-4cc7-9c2f-12d7daba09f7","Type":"ContainerDied","Data":"1724ff4f36d732d70ac40c589c9ef831231a76e5d7a3b62df79c6e1097650c48"}
Sep 11 16:39:14.832390 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.832190    2579 scope.go:117] "RemoveContainer" containerID="4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"
Sep 11 16:39:14.832630 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.832487    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:14.839876 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.839862    2579 scope.go:117] "RemoveContainer" containerID="b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"
Sep 11 16:39:14.846527 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.846512    2579 scope.go:117] "RemoveContainer" containerID="4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"
Sep 11 16:39:14.846779 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:14.846761    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039\": container with ID starting with 4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039 not found: ID does not exist" containerID="4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"
Sep 11 16:39:14.846827 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.846786    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039"} err="failed to get container status \"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039\": rpc error: code = NotFound desc = could not find container \"4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039\": container with ID starting with 4bfe4593ba3b8d47285f173f18c87a2787d5fddb71cc3375020a8b4f53f2c039 not found: ID does not exist"
Sep 11 16:39:14.846827 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.846807    2579 scope.go:117] "RemoveContainer" containerID="b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"
Sep 11 16:39:14.847060 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:14.847043    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00\": container with ID starting with b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00 not found: ID does not exist" containerID="b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"
Sep 11 16:39:14.847109 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.847069    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00"} err="failed to get container status \"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00\": rpc error: code = NotFound desc = could not find container \"b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00\": container with ID starting with b96363665123e5aee5ae5ec6cb91ca315a5dd17d440b65dd399ea421c455fb00 not found: ID does not exist"
Sep 11 16:39:14.850344 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.850322    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:39:14.855670 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:14.855650    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-a9581-predictor-5656556755-688q4"]
Sep 11 16:39:16.749999 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:16.749960    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" path="/var/lib/kubelet/pods/495ada4f-7c74-4cc7-9c2f-12d7daba09f7/volumes"
Sep 11 16:39:19.837684 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:19.837655    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:39:19.838107 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:19.838050    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:24.710240 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:24.710214    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:39:29.838629 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:29.838591    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:39.838603 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:39.838558    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:46.859157 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.859123    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:39:46.859698 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.859345    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container" containerID="cri-o://f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1" gracePeriod=30
Sep 11 16:39:46.859698 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.859424    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kube-rbac-proxy" containerID="cri-o://94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810" gracePeriod=30
Sep 11 16:39:46.987278 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987253    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:39:46.987545 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987533    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kube-rbac-proxy"
Sep 11 16:39:46.987595 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987547    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kube-rbac-proxy"
Sep 11 16:39:46.987595 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987564    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container"
Sep 11 16:39:46.987595 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987569    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container"
Sep 11 16:39:46.987685 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987614    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kserve-container"
Sep 11 16:39:46.987685 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.987621    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="495ada4f-7c74-4cc7-9c2f-12d7daba09f7" containerName="kube-rbac-proxy"
Sep 11 16:39:46.990557 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.990543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:46.993276 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.993257    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ea3c5-kube-rbac-proxy-sar-config\""
Sep 11 16:39:46.993998 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:46.993981    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ea3c5-predictor-serving-cert\""
Sep 11 16:39:47.004965 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.004945    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:39:47.075839 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.075805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4kx8\" (UniqueName: \"kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.076016 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.075857    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.076016 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.075924    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.177042 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.177009    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4kx8\" (UniqueName: \"kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.177218 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.177053    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.177218 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.177097    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.177337 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:47.177236    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-serving-cert: secret "success-200-isvc-ea3c5-predictor-serving-cert" not found
Sep 11 16:39:47.177337 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:47.177309    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls podName:cce281a8-aa85-484f-822a-4115ca0e68fa nodeName:}" failed. No retries permitted until 2026-09-11 16:39:47.67728807 +0000 UTC m=+1343.450528100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls") pod "success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" (UID: "cce281a8-aa85-484f-822a-4115ca0e68fa") : secret "success-200-isvc-ea3c5-predictor-serving-cert" not found
Sep 11 16:39:47.177865 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.177842    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.189530 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.189506    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4kx8\" (UniqueName: \"kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.682719 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.682688    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.685272 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.685240    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") pod \"success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") " pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.900396 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.900357    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:47.952787 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.952760    2579 generic.go:356] "Generic (PLEG): container finished" podID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerID="94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810" exitCode=2
Sep 11 16:39:47.952923 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:47.952788    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerDied","Data":"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"}
Sep 11 16:39:48.169742 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.169718    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:39:48.172238 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:39:48.172214    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcce281a8_aa85_484f_822a_4115ca0e68fa.slice/crio-57667f5dbc16d9bb50ab70f9cc44b2c038379415e1a5e6801ec777ea88508e6b WatchSource:0}: Error finding container 57667f5dbc16d9bb50ab70f9cc44b2c038379415e1a5e6801ec777ea88508e6b: Status 404 returned error can't find the container with id 57667f5dbc16d9bb50ab70f9cc44b2c038379415e1a5e6801ec777ea88508e6b
Sep 11 16:39:48.959190 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.959153    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerStarted","Data":"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"}
Sep 11 16:39:48.959632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.959209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerStarted","Data":"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"}
Sep 11 16:39:48.959632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.959238    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:48.959632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.959255    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerStarted","Data":"57667f5dbc16d9bb50ab70f9cc44b2c038379415e1a5e6801ec777ea88508e6b"}
Sep 11 16:39:48.959632 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.959286    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:48.960653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.960630    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:39:48.977012 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:48.976968    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podStartSLOduration=2.976951703 podStartE2EDuration="2.976951703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:39:48.975342962 +0000 UTC m=+1344.748582994" watchObservedRunningTime="2026-09-11 16:39:48.976951703 +0000 UTC m=+1344.750191732"
Sep 11 16:39:49.705159 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:49.705119    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused"
Sep 11 16:39:49.838849 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:49.838810    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused"
Sep 11 16:39:49.963472 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:49.963366    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:39:50.094869 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.094842    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:39:50.201815 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.201780    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") pod \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") "
Sep 11 16:39:50.201955 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.201848    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config\") pod \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") "
Sep 11 16:39:50.201955 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.201892    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn469\" (UniqueName: \"kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469\") pod \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\" (UID: \"d612dee1-fcd2-405b-bbae-a5ef4927efc2\") "
Sep 11 16:39:50.202173 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.202149    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "d612dee1-fcd2-405b-bbae-a5ef4927efc2" (UID: "d612dee1-fcd2-405b-bbae-a5ef4927efc2"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:39:50.204116 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.204086    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d612dee1-fcd2-405b-bbae-a5ef4927efc2" (UID: "d612dee1-fcd2-405b-bbae-a5ef4927efc2"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:39:50.204116 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.204102    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469" (OuterVolumeSpecName: "kube-api-access-rn469") pod "d612dee1-fcd2-405b-bbae-a5ef4927efc2" (UID: "d612dee1-fcd2-405b-bbae-a5ef4927efc2"). InnerVolumeSpecName "kube-api-access-rn469". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:39:50.303423 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.303340    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn469\" (UniqueName: \"kubernetes.io/projected/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-api-access-rn469\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.303423 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.303370    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d612dee1-fcd2-405b-bbae-a5ef4927efc2-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.303423 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.303382    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/d612dee1-fcd2-405b-bbae-a5ef4927efc2-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:39:50.969075 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.969042    2579 generic.go:356] "Generic (PLEG): container finished" podID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerID="f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1" exitCode=0
Sep 11 16:39:50.969516 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.969127    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"
Sep 11 16:39:50.969516 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.969125    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerDied","Data":"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"}
Sep 11 16:39:50.969516 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.969252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt" event={"ID":"d612dee1-fcd2-405b-bbae-a5ef4927efc2","Type":"ContainerDied","Data":"6d8284841a45ec135ca22d1987e541d93182ebee7de3dbc1cc1d5c17fdfc1812"}
Sep 11 16:39:50.969516 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.969277    2579 scope.go:117] "RemoveContainer" containerID="94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"
Sep 11 16:39:50.977506 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.977315    2579 scope.go:117] "RemoveContainer" containerID="f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"
Sep 11 16:39:50.984172 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.984147    2579 scope.go:117] "RemoveContainer" containerID="94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"
Sep 11 16:39:50.984435 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:50.984418    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810\": container with ID starting with 94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810 not found: ID does not exist" containerID="94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"
Sep 11 16:39:50.984508 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.984441    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810"} err="failed to get container status \"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810\": rpc error: code = NotFound desc = could not find container \"94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810\": container with ID starting with 94672646b5ebad34e3dba4b2fd34c846b0a4e19ec5fc7be41845b02f3178f810 not found: ID does not exist"
Sep 11 16:39:50.984508 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.984481    2579 scope.go:117] "RemoveContainer" containerID="f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"
Sep 11 16:39:50.984732 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:39:50.984714    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1\": container with ID starting with f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1 not found: ID does not exist" containerID="f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"
Sep 11 16:39:50.984782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.984741    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1"} err="failed to get container status \"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1\": rpc error: code = NotFound desc = could not find container \"f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1\": container with ID starting with f8f250f71cdd1b7a9c82e6a802df974949691823168d8bdc7e411bca221a72e1 not found: ID does not exist"
Sep 11 16:39:50.988656 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.988636    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:39:50.995227 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:50.995209    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e5408-predictor-765956d4cf-svlxt"]
Sep 11 16:39:52.749820 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:52.749788    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" path="/var/lib/kubelet/pods/d612dee1-fcd2-405b-bbae-a5ef4927efc2/volumes"
Sep 11 16:39:54.967607 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:54.967579    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:39:54.968016 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:54.967976    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:39:59.839300 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:39:59.839273    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:40:04.968668 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:40:04.968626    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:40:14.968238 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:40:14.968195    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:40:24.968587 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:40:24.968544    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:40:34.969069 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:40:34.969041    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:48:25.985810 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:25.985771    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:48:25.986330 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:25.985988    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container" containerID="cri-o://b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453" gracePeriod=30
Sep 11 16:48:25.986330 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:25.986059    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kube-rbac-proxy" containerID="cri-o://b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd" gracePeriod=30
Sep 11 16:48:26.076714 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.076681    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:48:26.077092 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077077    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container"
Sep 11 16:48:26.077141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077096    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container"
Sep 11 16:48:26.077141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077118    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kube-rbac-proxy"
Sep 11 16:48:26.077141 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077127    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kube-rbac-proxy"
Sep 11 16:48:26.077229 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077196    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kserve-container"
Sep 11 16:48:26.077229 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.077206    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d612dee1-fcd2-405b-bbae-a5ef4927efc2" containerName="kube-rbac-proxy"
Sep 11 16:48:26.080468 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.080438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.083768 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.083739    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ddc34-kube-rbac-proxy-sar-config\""
Sep 11 16:48:26.084089 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.084072    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-ddc34-predictor-serving-cert\""
Sep 11 16:48:26.093282 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.093255    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:48:26.224869 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.224816    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.225078 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.224933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbwgq\" (UniqueName: \"kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.225078 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.224969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.325635 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.325542    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbwgq\" (UniqueName: \"kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.325635 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.325592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.325849 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.325648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.325849 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:48:26.325758    2579 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-serving-cert: secret "success-200-isvc-ddc34-predictor-serving-cert" not found
Sep 11 16:48:26.325849 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:48:26.325832    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls podName:eebbd631-41d6-4bce-88a0-5958993b067f nodeName:}" failed. No retries permitted until 2026-09-11 16:48:26.82580908 +0000 UTC m=+1862.599049092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls") pod "success-200-isvc-ddc34-predictor-54b9f46b47-258vb" (UID: "eebbd631-41d6-4bce-88a0-5958993b067f") : secret "success-200-isvc-ddc34-predictor-serving-cert" not found
Sep 11 16:48:26.326417 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.326384    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.334827 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.334801    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbwgq\" (UniqueName: \"kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.772777 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.772744    2579 generic.go:356] "Generic (PLEG): container finished" podID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerID="b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd" exitCode=2
Sep 11 16:48:26.772777 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.772786    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerDied","Data":"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"}
Sep 11 16:48:26.829517 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.829476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.832128 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.832108    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") pod \"success-200-isvc-ddc34-predictor-54b9f46b47-258vb\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") " pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:26.991231 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:26.991196    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:27.263463 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.263410    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:48:27.266573 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:48:27.266533    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeebbd631_41d6_4bce_88a0_5958993b067f.slice/crio-a521e70a8efe7b40ed376ad1d9c78c081d9190d05bbb033920558ec813cb7f3d WatchSource:0}: Error finding container a521e70a8efe7b40ed376ad1d9c78c081d9190d05bbb033920558ec813cb7f3d: Status 404 returned error can't find the container with id a521e70a8efe7b40ed376ad1d9c78c081d9190d05bbb033920558ec813cb7f3d
Sep 11 16:48:27.778390 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.778351    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerStarted","Data":"19edc9d8c0cc26845599c4e8dc86c05786dcbfbe82642b3b483f640b8b6b8d3b"}
Sep 11 16:48:27.778576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.778409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerStarted","Data":"738cddfd026a826add49a7e94a06a0fc127960e3e0212f2bff1a56252bfa2c39"}
Sep 11 16:48:27.778576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.778435    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:27.778576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.778480    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:27.778576 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.778499    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerStarted","Data":"a521e70a8efe7b40ed376ad1d9c78c081d9190d05bbb033920558ec813cb7f3d"}
Sep 11 16:48:27.779945 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.779913    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:48:27.797397 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:27.797355    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podStartSLOduration=1.797342376 podStartE2EDuration="1.797342376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:48:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:48:27.796946893 +0000 UTC m=+1863.570186924" watchObservedRunningTime="2026-09-11 16:48:27.797342376 +0000 UTC m=+1863.570582409"
Sep 11 16:48:28.782242 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:28.782202    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:48:29.134694 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.134669    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:48:29.249567 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.249530    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pj5hz\" (UniqueName: \"kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz\") pod \"96ef3781-b318-49f5-a13d-f7cc49f05109\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") "
Sep 11 16:48:29.249739 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.249587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config\") pod \"96ef3781-b318-49f5-a13d-f7cc49f05109\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") "
Sep 11 16:48:29.249739 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.249607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") pod \"96ef3781-b318-49f5-a13d-f7cc49f05109\" (UID: \"96ef3781-b318-49f5-a13d-f7cc49f05109\") "
Sep 11 16:48:29.249955 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.249917    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "96ef3781-b318-49f5-a13d-f7cc49f05109" (UID: "96ef3781-b318-49f5-a13d-f7cc49f05109"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:48:29.251912 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.251881    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "96ef3781-b318-49f5-a13d-f7cc49f05109" (UID: "96ef3781-b318-49f5-a13d-f7cc49f05109"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:48:29.252002 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.251911    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz" (OuterVolumeSpecName: "kube-api-access-pj5hz") pod "96ef3781-b318-49f5-a13d-f7cc49f05109" (UID: "96ef3781-b318-49f5-a13d-f7cc49f05109"). InnerVolumeSpecName "kube-api-access-pj5hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:48:29.350580 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.350481    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:48:29.350580 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.350525    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96ef3781-b318-49f5-a13d-f7cc49f05109-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:48:29.350580 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.350536    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pj5hz\" (UniqueName: \"kubernetes.io/projected/96ef3781-b318-49f5-a13d-f7cc49f05109-kube-api-access-pj5hz\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:48:29.787671 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.787634    2579 generic.go:356] "Generic (PLEG): container finished" podID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerID="b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453" exitCode=0
Sep 11 16:48:29.788146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.787698    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"
Sep 11 16:48:29.788146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.787710    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerDied","Data":"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"}
Sep 11 16:48:29.788146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.787764    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m" event={"ID":"96ef3781-b318-49f5-a13d-f7cc49f05109","Type":"ContainerDied","Data":"27bdb565421cb3d63c5b530d0ff6984124ac6f16a6802b4149bd1b420ff2ea15"}
Sep 11 16:48:29.788146 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.787791    2579 scope.go:117] "RemoveContainer" containerID="b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"
Sep 11 16:48:29.796533 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.796511    2579 scope.go:117] "RemoveContainer" containerID="b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"
Sep 11 16:48:29.803971 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.803951    2579 scope.go:117] "RemoveContainer" containerID="b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"
Sep 11 16:48:29.804260 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:48:29.804242    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd\": container with ID starting with b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd not found: ID does not exist" containerID="b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"
Sep 11 16:48:29.804330 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.804273    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd"} err="failed to get container status \"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd\": rpc error: code = NotFound desc = could not find container \"b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd\": container with ID starting with b94be0f30a1fab32b2522108b985ed5c108f1cab59d6918984bd7805b25068cd not found: ID does not exist"
Sep 11 16:48:29.804330 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.804297    2579 scope.go:117] "RemoveContainer" containerID="b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"
Sep 11 16:48:29.804593 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:48:29.804576    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453\": container with ID starting with b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453 not found: ID does not exist" containerID="b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"
Sep 11 16:48:29.804639 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.804599    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453"} err="failed to get container status \"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453\": rpc error: code = NotFound desc = could not find container \"b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453\": container with ID starting with b735c8455a7b8687f2db9307672a25d796692038790cb03c37610f096c357453 not found: ID does not exist"
Sep 11 16:48:29.808724 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.808700    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:48:29.810917 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:29.810896    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1ab9e-predictor-595c484fbc-shn4m"]
Sep 11 16:48:30.749550 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:30.749417    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" path="/var/lib/kubelet/pods/96ef3781-b318-49f5-a13d-f7cc49f05109/volumes"
Sep 11 16:48:33.786993 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:33.786964    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:48:33.787496 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:33.787469    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:48:43.787815 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:43.787768    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:48:53.787749 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:48:53.787714    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:49:01.711793 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.711763    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:49:01.712328 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.711980    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" containerID="cri-o://40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a" gracePeriod=30
Sep 11 16:49:01.712328 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.712084    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kube-rbac-proxy" containerID="cri-o://9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55" gracePeriod=30
Sep 11 16:49:01.837764 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.837728    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 16:49:01.837998 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.837987    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kube-rbac-proxy"
Sep 11 16:49:01.838041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.838001    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kube-rbac-proxy"
Sep 11 16:49:01.838041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.838013    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container"
Sep 11 16:49:01.838041 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.838019    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container"
Sep 11 16:49:01.838134 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.838065    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kserve-container"
Sep 11 16:49:01.838134 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.838074    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef3781-b318-49f5-a13d-f7cc49f05109" containerName="kube-rbac-proxy"
Sep 11 16:49:01.840914 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.840900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.843396 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.843374    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-482a1-kube-rbac-proxy-sar-config\""
Sep 11 16:49:01.843561 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.843542    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-482a1-predictor-serving-cert\""
Sep 11 16:49:01.850509 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.850486    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 16:49:01.885225 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.885189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q48dj\" (UniqueName: \"kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.885225 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.885228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.885477 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.885255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.910203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.910174    2579 generic.go:356] "Generic (PLEG): container finished" podID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerID="9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55" exitCode=2
Sep 11 16:49:01.910384 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.910230    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerDied","Data":"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"}
Sep 11 16:49:01.986609 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.986520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q48dj\" (UniqueName: \"kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.986609 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.986571    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.986807 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.986614    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.987242 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.987223    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:01.989367 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:01.989342    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:02.004985 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.004954    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q48dj\" (UniqueName: \"kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj\") pod \"success-200-isvc-482a1-predictor-747c4d74f5-fq6rz\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") " pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:02.151921 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.151879    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:02.425808 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.425784    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 16:49:02.428365 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:49:02.428337    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5018e8_bd51_4c7b_a22e_08d79473152e.slice/crio-5a6b0d154c9c01d4dbb0790a22bfdf2723700df11ba3444b2a14ae0418ebb6dc WatchSource:0}: Error finding container 5a6b0d154c9c01d4dbb0790a22bfdf2723700df11ba3444b2a14ae0418ebb6dc: Status 404 returned error can't find the container with id 5a6b0d154c9c01d4dbb0790a22bfdf2723700df11ba3444b2a14ae0418ebb6dc
Sep 11 16:49:02.921279 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.921239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerStarted","Data":"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"}
Sep 11 16:49:02.921703 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.921345    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerStarted","Data":"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"}
Sep 11 16:49:02.921703 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.921380    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:02.921703 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.921405    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerStarted","Data":"5a6b0d154c9c01d4dbb0790a22bfdf2723700df11ba3444b2a14ae0418ebb6dc"}
Sep 11 16:49:02.921703 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.921434    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:02.922756 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.922731    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:02.938254 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:02.938200    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podStartSLOduration=1.938185463 podStartE2EDuration="1.938185463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:02.937305556 +0000 UTC m=+1898.710545586" watchObservedRunningTime="2026-09-11 16:49:02.938185463 +0000 UTC m=+1898.711425494"
Sep 11 16:49:03.788327 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:03.788286    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.30:8080: connect: connection refused"
Sep 11 16:49:03.925305 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:03.925261    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:04.964180 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:04.964140    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused"
Sep 11 16:49:04.968307 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:04.968279    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused"
Sep 11 16:49:05.156412 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.156388    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:49:05.211294 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.211261    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4kx8\" (UniqueName: \"kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8\") pod \"cce281a8-aa85-484f-822a-4115ca0e68fa\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") "
Sep 11 16:49:05.211515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.211332    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config\") pod \"cce281a8-aa85-484f-822a-4115ca0e68fa\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") "
Sep 11 16:49:05.211515 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.211361    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") pod \"cce281a8-aa85-484f-822a-4115ca0e68fa\" (UID: \"cce281a8-aa85-484f-822a-4115ca0e68fa\") "
Sep 11 16:49:05.211782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.211754    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cce281a8-aa85-484f-822a-4115ca0e68fa" (UID: "cce281a8-aa85-484f-822a-4115ca0e68fa"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:49:05.213655 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.213632    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cce281a8-aa85-484f-822a-4115ca0e68fa" (UID: "cce281a8-aa85-484f-822a-4115ca0e68fa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:49:05.213738 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.213717    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8" (OuterVolumeSpecName: "kube-api-access-b4kx8") pod "cce281a8-aa85-484f-822a-4115ca0e68fa" (UID: "cce281a8-aa85-484f-822a-4115ca0e68fa"). InnerVolumeSpecName "kube-api-access-b4kx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:05.312383 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.312338    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.312383 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.312376    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cce281a8-aa85-484f-822a-4115ca0e68fa-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.312383 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.312387    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4kx8\" (UniqueName: \"kubernetes.io/projected/cce281a8-aa85-484f-822a-4115ca0e68fa-kube-api-access-b4kx8\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:05.933203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.933169    2579 generic.go:356] "Generic (PLEG): container finished" podID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerID="40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a" exitCode=0
Sep 11 16:49:05.933362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.933219    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"
Sep 11 16:49:05.933362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.933252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerDied","Data":"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"}
Sep 11 16:49:05.933362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.933320    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh" event={"ID":"cce281a8-aa85-484f-822a-4115ca0e68fa","Type":"ContainerDied","Data":"57667f5dbc16d9bb50ab70f9cc44b2c038379415e1a5e6801ec777ea88508e6b"}
Sep 11 16:49:05.933362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.933360    2579 scope.go:117] "RemoveContainer" containerID="9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"
Sep 11 16:49:05.941782 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.941766    2579 scope.go:117] "RemoveContainer" containerID="40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"
Sep 11 16:49:05.948847 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.948832    2579 scope.go:117] "RemoveContainer" containerID="9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"
Sep 11 16:49:05.949115 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:49:05.949099    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55\": container with ID starting with 9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55 not found: ID does not exist" containerID="9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"
Sep 11 16:49:05.949177 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.949121    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55"} err="failed to get container status \"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55\": rpc error: code = NotFound desc = could not find container \"9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55\": container with ID starting with 9bd987f5fe6309d23506f68bf87bb72ea117aab1304d06eadd7c795ed9ec9a55 not found: ID does not exist"
Sep 11 16:49:05.949177 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.949136    2579 scope.go:117] "RemoveContainer" containerID="40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"
Sep 11 16:49:05.949363 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:49:05.949344    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a\": container with ID starting with 40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a not found: ID does not exist" containerID="40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"
Sep 11 16:49:05.949403 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.949370    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a"} err="failed to get container status \"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a\": rpc error: code = NotFound desc = could not find container \"40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a\": container with ID starting with 40b6931b175d34402ec693b42d088174ae7ba6e0c78889a877f5473af9fa656a not found: ID does not exist"
Sep 11 16:49:05.953838 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.953814    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:49:05.957522 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:05.957505    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ea3c5-predictor-86955dc77c-sd4rh"]
Sep 11 16:49:06.749727 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:06.749693    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" path="/var/lib/kubelet/pods/cce281a8-aa85-484f-822a-4115ca0e68fa/volumes"
Sep 11 16:49:08.931129 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:08.931100    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:08.931669 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:08.931646    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:13.788881 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:13.788852    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:49:18.931988 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:18.931944    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:28.931929 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:28.931886    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:36.277220 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.277184    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:49:36.277690 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.277419    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container" containerID="cri-o://738cddfd026a826add49a7e94a06a0fc127960e3e0212f2bff1a56252bfa2c39" gracePeriod=30
Sep 11 16:49:36.277690 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.277491    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kube-rbac-proxy" containerID="cri-o://19edc9d8c0cc26845599c4e8dc86c05786dcbfbe82642b3b483f640b8b6b8d3b" gracePeriod=30
Sep 11 16:49:36.409653 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.409612    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:49:36.410071 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410056    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container"
Sep 11 16:49:36.410117 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410075    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container"
Sep 11 16:49:36.410117 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410105    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kube-rbac-proxy"
Sep 11 16:49:36.410117 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410113    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kube-rbac-proxy"
Sep 11 16:49:36.410210 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410172    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kube-rbac-proxy"
Sep 11 16:49:36.410210 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.410183    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cce281a8-aa85-484f-822a-4115ca0e68fa" containerName="kserve-container"
Sep 11 16:49:36.413520 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.413495    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.415912 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.415890    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-20f8f-predictor-serving-cert\""
Sep 11 16:49:36.416025 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.416006    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-20f8f-kube-rbac-proxy-sar-config\""
Sep 11 16:49:36.422759 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.422729    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:49:36.558119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.558008    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.558119 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.558108    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.558362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.558143    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dqjs\" (UniqueName: \"kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.659672 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.659626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.659871 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.659724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.659871 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.659758    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dqjs\" (UniqueName: \"kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.660362 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.660336    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.662429 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.662404    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.667948 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.667913    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dqjs\" (UniqueName: \"kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs\") pod \"success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") " pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:36.724549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:36.724504    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:37.003115 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:37.002980    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:49:37.006007 ip-10-0-136-18 kubenswrapper[2579]: W0911 16:49:37.005980    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075be758_ad85_458e_bf71_934e5a2f06a6.slice/crio-b724134a9e8c7b7091594cd8fa1a174a99f3fe34bb03751fa16a0abcf765090b WatchSource:0}: Error finding container b724134a9e8c7b7091594cd8fa1a174a99f3fe34bb03751fa16a0abcf765090b: Status 404 returned error can't find the container with id b724134a9e8c7b7091594cd8fa1a174a99f3fe34bb03751fa16a0abcf765090b
Sep 11 16:49:37.047067 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:37.047023    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerStarted","Data":"b724134a9e8c7b7091594cd8fa1a174a99f3fe34bb03751fa16a0abcf765090b"}
Sep 11 16:49:37.049055 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:37.049025    2579 generic.go:356] "Generic (PLEG): container finished" podID="eebbd631-41d6-4bce-88a0-5958993b067f" containerID="19edc9d8c0cc26845599c4e8dc86c05786dcbfbe82642b3b483f640b8b6b8d3b" exitCode=2
Sep 11 16:49:37.049203 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:37.049091    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerDied","Data":"19edc9d8c0cc26845599c4e8dc86c05786dcbfbe82642b3b483f640b8b6b8d3b"}
Sep 11 16:49:38.055217 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.055175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerStarted","Data":"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"}
Sep 11 16:49:38.055746 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.055251    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerStarted","Data":"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"}
Sep 11 16:49:38.055746 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.055361    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:38.080366 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.079124    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podStartSLOduration=2.079097994 podStartE2EDuration="2.079097994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 16:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 16:49:38.075975864 +0000 UTC m=+1933.849215895" watchObservedRunningTime="2026-09-11 16:49:38.079097994 +0000 UTC m=+1933.852338026"
Sep 11 16:49:38.783107 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.783065    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.30:8643/healthz\": dial tcp 10.134.0.30:8643: connect: connection refused"
Sep 11 16:49:38.932382 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:38.932334    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Sep 11 16:49:39.059779 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:39.059682    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:39.061167 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:39.061135    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:49:40.066558 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.066523    2579 generic.go:356] "Generic (PLEG): container finished" podID="eebbd631-41d6-4bce-88a0-5958993b067f" containerID="738cddfd026a826add49a7e94a06a0fc127960e3e0212f2bff1a56252bfa2c39" exitCode=0
Sep 11 16:49:40.067055 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.066599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerDied","Data":"738cddfd026a826add49a7e94a06a0fc127960e3e0212f2bff1a56252bfa2c39"}
Sep 11 16:49:40.067268 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.067243    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:49:40.117814 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.117788    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:49:40.191145 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.191112    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") pod \"eebbd631-41d6-4bce-88a0-5958993b067f\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") "
Sep 11 16:49:40.191359 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.191189    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config\") pod \"eebbd631-41d6-4bce-88a0-5958993b067f\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") "
Sep 11 16:49:40.191359 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.191300    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbwgq\" (UniqueName: \"kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq\") pod \"eebbd631-41d6-4bce-88a0-5958993b067f\" (UID: \"eebbd631-41d6-4bce-88a0-5958993b067f\") "
Sep 11 16:49:40.191593 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.191568    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "eebbd631-41d6-4bce-88a0-5958993b067f" (UID: "eebbd631-41d6-4bce-88a0-5958993b067f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:49:40.191684 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.191665    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/eebbd631-41d6-4bce-88a0-5958993b067f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:40.193662 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.193635    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "eebbd631-41d6-4bce-88a0-5958993b067f" (UID: "eebbd631-41d6-4bce-88a0-5958993b067f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:49:40.193662 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.193656    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq" (OuterVolumeSpecName: "kube-api-access-mbwgq") pod "eebbd631-41d6-4bce-88a0-5958993b067f" (UID: "eebbd631-41d6-4bce-88a0-5958993b067f"). InnerVolumeSpecName "kube-api-access-mbwgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:49:40.292722 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.292628    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbwgq\" (UniqueName: \"kubernetes.io/projected/eebbd631-41d6-4bce-88a0-5958993b067f-kube-api-access-mbwgq\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:40.292722 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:40.292660    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/eebbd631-41d6-4bce-88a0-5958993b067f-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:49:41.073521 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.073412    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"
Sep 11 16:49:41.073521 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.073424    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb" event={"ID":"eebbd631-41d6-4bce-88a0-5958993b067f","Type":"ContainerDied","Data":"a521e70a8efe7b40ed376ad1d9c78c081d9190d05bbb033920558ec813cb7f3d"}
Sep 11 16:49:41.073521 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.073507    2579 scope.go:117] "RemoveContainer" containerID="19edc9d8c0cc26845599c4e8dc86c05786dcbfbe82642b3b483f640b8b6b8d3b"
Sep 11 16:49:41.082102 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.082081    2579 scope.go:117] "RemoveContainer" containerID="738cddfd026a826add49a7e94a06a0fc127960e3e0212f2bff1a56252bfa2c39"
Sep 11 16:49:41.091338 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.091309    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:49:41.096393 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:41.096367    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-ddc34-predictor-54b9f46b47-258vb"]
Sep 11 16:49:42.750213 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:42.750172    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" path="/var/lib/kubelet/pods/eebbd631-41d6-4bce-88a0-5958993b067f/volumes"
Sep 11 16:49:45.071505 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:45.071467    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:49:45.072162 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:45.072129    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:49:48.933240 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:48.933208    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 16:49:55.072394 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:49:55.072348    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:50:05.072543 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:50:05.072505    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:50:15.072092 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:50:15.072053    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Sep 11 16:50:25.072556 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:50:25.072524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:58:51.118737 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:51.118699    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:58:51.121107 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:51.118960    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container" containerID="cri-o://5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c" gracePeriod=30
Sep 11 16:58:51.121107 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:51.119000    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kube-rbac-proxy" containerID="cri-o://92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62" gracePeriod=30
Sep 11 16:58:52.021270 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:52.021236    2579 generic.go:356] "Generic (PLEG): container finished" podID="075be758-ad85-458e-bf71-934e5a2f06a6" containerID="92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62" exitCode=2
Sep 11 16:58:52.021428 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:52.021293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerDied","Data":"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"}
Sep 11 16:58:53.652296 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.652275    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:58:53.655366 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.655350    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls\") pod \"075be758-ad85-458e-bf71-934e5a2f06a6\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") "
Sep 11 16:58:53.655437 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.655426    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config\") pod \"075be758-ad85-458e-bf71-934e5a2f06a6\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") "
Sep 11 16:58:53.655510 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.655460    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dqjs\" (UniqueName: \"kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs\") pod \"075be758-ad85-458e-bf71-934e5a2f06a6\" (UID: \"075be758-ad85-458e-bf71-934e5a2f06a6\") "
Sep 11 16:58:53.655801 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.655773    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "075be758-ad85-458e-bf71-934e5a2f06a6" (UID: "075be758-ad85-458e-bf71-934e5a2f06a6"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 16:58:53.657415 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.657391    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "075be758-ad85-458e-bf71-934e5a2f06a6" (UID: "075be758-ad85-458e-bf71-934e5a2f06a6"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 16:58:53.657549 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.657523    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs" (OuterVolumeSpecName: "kube-api-access-2dqjs") pod "075be758-ad85-458e-bf71-934e5a2f06a6" (UID: "075be758-ad85-458e-bf71-934e5a2f06a6"). InnerVolumeSpecName "kube-api-access-2dqjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 16:58:53.756392 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.756334    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/075be758-ad85-458e-bf71-934e5a2f06a6-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:58:53.756392 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.756358    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dqjs\" (UniqueName: \"kubernetes.io/projected/075be758-ad85-458e-bf71-934e5a2f06a6-kube-api-access-2dqjs\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:58:53.756392 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:53.756368    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/075be758-ad85-458e-bf71-934e5a2f06a6-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 16:58:54.029819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.029749    2579 generic.go:356] "Generic (PLEG): container finished" podID="075be758-ad85-458e-bf71-934e5a2f06a6" containerID="5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c" exitCode=0
Sep 11 16:58:54.029819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.029794    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerDied","Data":"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"}
Sep 11 16:58:54.029819 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.029815    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"
Sep 11 16:58:54.030003 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.029858    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4" event={"ID":"075be758-ad85-458e-bf71-934e5a2f06a6","Type":"ContainerDied","Data":"b724134a9e8c7b7091594cd8fa1a174a99f3fe34bb03751fa16a0abcf765090b"}
Sep 11 16:58:54.030003 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.029882    2579 scope.go:117] "RemoveContainer" containerID="92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"
Sep 11 16:58:54.037738 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.037725    2579 scope.go:117] "RemoveContainer" containerID="5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"
Sep 11 16:58:54.044577 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.044536    2579 scope.go:117] "RemoveContainer" containerID="92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"
Sep 11 16:58:54.044778 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:58:54.044761    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62\": container with ID starting with 92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62 not found: ID does not exist" containerID="92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"
Sep 11 16:58:54.044816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.044785    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62"} err="failed to get container status \"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62\": rpc error: code = NotFound desc = could not find container \"92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62\": container with ID starting with 92ac0b7a8b571b98c6e62c886adc351f1e04d3d1f20b5e2b0d621932a9844f62 not found: ID does not exist"
Sep 11 16:58:54.044816 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.044801    2579 scope.go:117] "RemoveContainer" containerID="5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"
Sep 11 16:58:54.045030 ip-10-0-136-18 kubenswrapper[2579]: E0911 16:58:54.045013    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c\": container with ID starting with 5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c not found: ID does not exist" containerID="5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"
Sep 11 16:58:54.045079 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.045034    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c"} err="failed to get container status \"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c\": rpc error: code = NotFound desc = could not find container \"5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c\": container with ID starting with 5f7872f756149bb319926ef678c0654b1dd633c14df1816a62a3c661560ebd7c not found: ID does not exist"
Sep 11 16:58:54.050396 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.050375    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:58:54.053894 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.053872    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-20f8f-predictor-7996fdcc9f-82nm4"]
Sep 11 16:58:54.749343 ip-10-0-136-18 kubenswrapper[2579]: I0911 16:58:54.749313    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" path="/var/lib/kubelet/pods/075be758-ad85-458e-bf71-934e5a2f06a6/volumes"
Sep 11 17:06:21.125407 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:21.125370    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 17:06:21.127950 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:21.125708    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container" containerID="cri-o://15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237" gracePeriod=30
Sep 11 17:06:21.127950 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:21.125779    2579 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kube-rbac-proxy" containerID="cri-o://ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f" gracePeriod=30
Sep 11 17:06:21.155005 ip-10-0-136-18 kubenswrapper[2579]: E0911 17:06:21.154966    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5018e8_bd51_4c7b_a22e_08d79473152e.slice/crio-ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5018e8_bd51_4c7b_a22e_08d79473152e.slice/crio-conmon-ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 17:06:21.585910 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:21.585828    2579 generic.go:356] "Generic (PLEG): container finished" podID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerID="ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f" exitCode=2
Sep 11 17:06:21.585910 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:21.585901    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerDied","Data":"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"}
Sep 11 17:06:23.926079 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:23.926034    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.31:8643/healthz\": dial tcp 10.134.0.31:8643: connect: connection refused"
Sep 11 17:06:24.164021 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.163995    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 17:06:24.248323 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.248242    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q48dj\" (UniqueName: \"kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj\") pod \"ac5018e8-bd51-4c7b-a22e-08d79473152e\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") "
Sep 11 17:06:24.248323 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.248308    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config\") pod \"ac5018e8-bd51-4c7b-a22e-08d79473152e\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") "
Sep 11 17:06:24.248527 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.248336    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls\") pod \"ac5018e8-bd51-4c7b-a22e-08d79473152e\" (UID: \"ac5018e8-bd51-4c7b-a22e-08d79473152e\") "
Sep 11 17:06:24.248769 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.248746    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ac5018e8-bd51-4c7b-a22e-08d79473152e" (UID: "ac5018e8-bd51-4c7b-a22e-08d79473152e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:06:24.250512 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.250494    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ac5018e8-bd51-4c7b-a22e-08d79473152e" (UID: "ac5018e8-bd51-4c7b-a22e-08d79473152e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:06:24.250620 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.250596    2579 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj" (OuterVolumeSpecName: "kube-api-access-q48dj") pod "ac5018e8-bd51-4c7b-a22e-08d79473152e" (UID: "ac5018e8-bd51-4c7b-a22e-08d79473152e"). InnerVolumeSpecName "kube-api-access-q48dj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:06:24.349347 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.349314    2579 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.349347 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.349341    2579 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ac5018e8-bd51-4c7b-a22e-08d79473152e-proxy-tls\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.349347 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.349352    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q48dj\" (UniqueName: \"kubernetes.io/projected/ac5018e8-bd51-4c7b-a22e-08d79473152e-kube-api-access-q48dj\") on node \"ip-10-0-136-18.ec2.internal\" DevicePath \"\""
Sep 11 17:06:24.598494 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.598390    2579 generic.go:356] "Generic (PLEG): container finished" podID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerID="15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237" exitCode=0
Sep 11 17:06:24.598494 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.598472    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"
Sep 11 17:06:24.598666 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.598474    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerDied","Data":"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"}
Sep 11 17:06:24.598666 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.598614    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz" event={"ID":"ac5018e8-bd51-4c7b-a22e-08d79473152e","Type":"ContainerDied","Data":"5a6b0d154c9c01d4dbb0790a22bfdf2723700df11ba3444b2a14ae0418ebb6dc"}
Sep 11 17:06:24.598666 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.598641    2579 scope.go:117] "RemoveContainer" containerID="ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"
Sep 11 17:06:24.606999 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.606984    2579 scope.go:117] "RemoveContainer" containerID="15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"
Sep 11 17:06:24.614300 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.614278    2579 scope.go:117] "RemoveContainer" containerID="ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"
Sep 11 17:06:24.614705 ip-10-0-136-18 kubenswrapper[2579]: E0911 17:06:24.614682    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f\": container with ID starting with ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f not found: ID does not exist" containerID="ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"
Sep 11 17:06:24.614821 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.614713    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f"} err="failed to get container status \"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f\": rpc error: code = NotFound desc = could not find container \"ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f\": container with ID starting with ef7a921e1d681fafc12b237161f038627aa1e7138e9b09f58fed6018172a389f not found: ID does not exist"
Sep 11 17:06:24.614821 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.614735    2579 scope.go:117] "RemoveContainer" containerID="15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"
Sep 11 17:06:24.615022 ip-10-0-136-18 kubenswrapper[2579]: E0911 17:06:24.615005    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237\": container with ID starting with 15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237 not found: ID does not exist" containerID="15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"
Sep 11 17:06:24.615061 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.615027    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237"} err="failed to get container status \"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237\": rpc error: code = NotFound desc = could not find container \"15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237\": container with ID starting with 15734c381b00ff44886fa446bc8e16b6a36da1698c261038d1b7106671078237 not found: ID does not exist"
Sep 11 17:06:24.618906 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.618886    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 17:06:24.624387 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.624364    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-482a1-predictor-747c4d74f5-fq6rz"]
Sep 11 17:06:24.749341 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:24.749297    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" path="/var/lib/kubelet/pods/ac5018e8-bd51-4c7b-a22e-08d79473152e/volumes"
Sep 11 17:06:54.133338 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133306    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8n4t/must-gather-z26d5"]
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133567    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133578    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133587    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133592    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133598    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133604    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133611    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133616    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133626    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133631    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133646    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133650    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133687    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133695    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133701    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133707    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5018e8-bd51-4c7b-a22e-08d79473152e" containerName="kube-rbac-proxy"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133713    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="075be758-ad85-458e-bf71-934e5a2f06a6" containerName="kserve-container"
Sep 11 17:06:54.133813 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.133719    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="eebbd631-41d6-4bce-88a0-5958993b067f" containerName="kserve-container"
Sep 11 17:06:54.136407 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.136390    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.138657 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.138634    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"kube-root-ca.crt\""
Sep 11 17:06:54.138848 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.138688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g8n4t\"/\"openshift-service-ca.crt\""
Sep 11 17:06:54.138930 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.138892    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g8n4t\"/\"default-dockercfg-rjl6t\""
Sep 11 17:06:54.145005 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.144982    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/must-gather-z26d5"]
Sep 11 17:06:54.282394 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.282350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v976\" (UniqueName: \"kubernetes.io/projected/b226d969-d352-4748-a5ce-76e54e958795-kube-api-access-8v976\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.282612 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.282421    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b226d969-d352-4748-a5ce-76e54e958795-must-gather-output\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.383834 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.383737    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b226d969-d352-4748-a5ce-76e54e958795-must-gather-output\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.383834 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.383784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v976\" (UniqueName: \"kubernetes.io/projected/b226d969-d352-4748-a5ce-76e54e958795-kube-api-access-8v976\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.384103 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.384082    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b226d969-d352-4748-a5ce-76e54e958795-must-gather-output\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.392258 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.392225    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v976\" (UniqueName: \"kubernetes.io/projected/b226d969-d352-4748-a5ce-76e54e958795-kube-api-access-8v976\") pod \"must-gather-z26d5\" (UID: \"b226d969-d352-4748-a5ce-76e54e958795\") " pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.446110 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.446064    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/must-gather-z26d5"
Sep 11 17:06:54.712985 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.712962    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/must-gather-z26d5"]
Sep 11 17:06:54.717902 ip-10-0-136-18 kubenswrapper[2579]: W0911 17:06:54.717667    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb226d969_d352_4748_a5ce_76e54e958795.slice/crio-7eab4ce2a06d1a4b3dad9273cf9fc6bce4f5a5011a3c95497d1ceee35ecb34ce WatchSource:0}: Error finding container 7eab4ce2a06d1a4b3dad9273cf9fc6bce4f5a5011a3c95497d1ceee35ecb34ce: Status 404 returned error can't find the container with id 7eab4ce2a06d1a4b3dad9273cf9fc6bce4f5a5011a3c95497d1ceee35ecb34ce
Sep 11 17:06:54.719228 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:54.719210    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:06:55.713826 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:55.713748    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/must-gather-z26d5" event={"ID":"b226d969-d352-4748-a5ce-76e54e958795","Type":"ContainerStarted","Data":"eda2c504c9888eaf2b3de9eeedf9a0596059844a5e909fd034a4becaa438a066"}
Sep 11 17:06:55.713826 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:55.713792    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/must-gather-z26d5" event={"ID":"b226d969-d352-4748-a5ce-76e54e958795","Type":"ContainerStarted","Data":"dddabedd297ffc9e7fc2956b2bb8c73033fa2d17af2058617056de56490642aa"}
Sep 11 17:06:55.713826 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:55.713805    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/must-gather-z26d5" event={"ID":"b226d969-d352-4748-a5ce-76e54e958795","Type":"ContainerStarted","Data":"7eab4ce2a06d1a4b3dad9273cf9fc6bce4f5a5011a3c95497d1ceee35ecb34ce"}
Sep 11 17:06:55.730710 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:55.730662    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8n4t/must-gather-z26d5" podStartSLOduration=1.018907164 podStartE2EDuration="1.730646265s" totalImagesPullingTime="711.739101ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:06:55.7282276 +0000 UTC m=+2971.501467632" watchObservedRunningTime="2026-09-11 17:06:55.730646265 +0000 UTC m=+2971.503886295"
Sep 11 17:06:56.741236 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:56.741210    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ft8q5_d2003fb4-89c4-4059-8096-cc104438d87c/global-pull-secret-syncer/0.log"
Sep 11 17:06:56.933219 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:56.933188    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tcclt_e56aab3b-bf82-4e9d-915f-b73e11e5ac5b/konnectivity-agent/0.log"
Sep 11 17:06:56.996159 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:06:56.996082    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-18.ec2.internal_34bdb01b23407b83abb142f3983543d1/haproxy/0.log"
Sep 11 17:07:01.852363 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:01.852341    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcl2g_a5e54a6c-de38-4b8a-8d40-c96668e58957/node-exporter/0.log"
Sep 11 17:07:01.870594 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:01.870549    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcl2g_a5e54a6c-de38-4b8a-8d40-c96668e58957/kube-rbac-proxy/0.log"
Sep 11 17:07:01.889614 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:01.889582    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcl2g_a5e54a6c-de38-4b8a-8d40-c96668e58957/init-textfile/0.log"
Sep 11 17:07:04.439764 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:04.439713    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6d44f4bcfc-xvx62_43c7f3e7-076d-410d-a329-31da4b0d75b1/console/0.log"
Sep 11 17:07:05.026667 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.026632    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"]
Sep 11 17:07:05.033071 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.033044    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.037673 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.037649    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"]
Sep 11 17:07:05.180642 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.180606    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-sys\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.180642 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.180650    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-podres\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.180892 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.180670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-proc\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.180892 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.180732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-lib-modules\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.180892 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.180774    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdqkt\" (UniqueName: \"kubernetes.io/projected/e76c0e7b-a1de-4000-b9ab-27871b59848a-kube-api-access-bdqkt\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281360 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281261    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-lib-modules\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281360 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281314    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdqkt\" (UniqueName: \"kubernetes.io/projected/e76c0e7b-a1de-4000-b9ab-27871b59848a-kube-api-access-bdqkt\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281360 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-sys\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281366    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-podres\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281391    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-proc\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281482    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-lib-modules\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281508    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-sys\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281486    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-proc\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.281689 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.281533    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e76c0e7b-a1de-4000-b9ab-27871b59848a-podres\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.290234 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.290173    2579 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdqkt\" (UniqueName: \"kubernetes.io/projected/e76c0e7b-a1de-4000-b9ab-27871b59848a-kube-api-access-bdqkt\") pod \"perf-node-gather-daemonset-7kk6w\" (UID: \"e76c0e7b-a1de-4000-b9ab-27871b59848a\") " pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.346741 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.346514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:05.640412 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.640385    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"]
Sep 11 17:07:05.643486 ip-10-0-136-18 kubenswrapper[2579]: W0911 17:07:05.643407    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode76c0e7b_a1de_4000_b9ab_27871b59848a.slice/crio-a71dd4a31e1844bbaba4d03cd72470b47a348e23a9b3686654d9d5f183f3e0fe WatchSource:0}: Error finding container a71dd4a31e1844bbaba4d03cd72470b47a348e23a9b3686654d9d5f183f3e0fe: Status 404 returned error can't find the container with id a71dd4a31e1844bbaba4d03cd72470b47a348e23a9b3686654d9d5f183f3e0fe
Sep 11 17:07:05.764172 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.764130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w" event={"ID":"e76c0e7b-a1de-4000-b9ab-27871b59848a","Type":"ContainerStarted","Data":"a71dd4a31e1844bbaba4d03cd72470b47a348e23a9b3686654d9d5f183f3e0fe"}
Sep 11 17:07:05.887129 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.887099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2g87_3d537be6-ce63-4191-9b02-1accc8b6f1d7/dns/0.log"
Sep 11 17:07:05.903376 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.903310    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-n2g87_3d537be6-ce63-4191-9b02-1accc8b6f1d7/kube-rbac-proxy/0.log"
Sep 11 17:07:05.940472 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:05.940430    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-knxgq_a4bbaa63-55aa-4948-8f3c-587253f132af/dns-node-resolver/0.log"
Sep 11 17:07:06.404651 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:06.404618    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6cbcdf84f-g8gp4_d9dcd92b-6438-41d1-8180-d84ec5fdf475/registry/0.log"
Sep 11 17:07:06.460106 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:06.460063    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zjzhz_d2c8df0b-55b4-4fc4-b451-5e99cfe694ff/node-ca/0.log"
Sep 11 17:07:06.773435 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:06.773344    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w" event={"ID":"e76c0e7b-a1de-4000-b9ab-27871b59848a","Type":"ContainerStarted","Data":"a7310bc3bc9c5ef925581e48e2fb243f475277833b3c6db69de64fbf96e560e8"}
Sep 11 17:07:06.773897 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:06.773646    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:06.794182 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:06.794127    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w" podStartSLOduration=1.794099662 podStartE2EDuration="1.794099662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:07:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:07:06.791196338 +0000 UTC m=+2982.564436370" watchObservedRunningTime="2026-09-11 17:07:06.794099662 +0000 UTC m=+2982.567339708"
Sep 11 17:07:07.440022 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:07.439994    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4qgsv_1cbdac5e-efe7-4a77-981a-ea193e51a9fb/serve-healthcheck-canary/0.log"
Sep 11 17:07:07.797743 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:07.797675    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m82tz_2c3f8709-56c0-4c3a-887b-2b74dfe217de/kube-rbac-proxy/0.log"
Sep 11 17:07:07.813208 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:07.813184    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m82tz_2c3f8709-56c0-4c3a-887b-2b74dfe217de/exporter/0.log"
Sep 11 17:07:07.829564 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:07.829531    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m82tz_2c3f8709-56c0-4c3a-887b-2b74dfe217de/extractor/0.log"
Sep 11 17:07:10.213278 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:10.213252    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-68ptc_e17e0022-1827-408f-8f59-21dbd6d68acf/s3-init/0.log"
Sep 11 17:07:10.226190 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:10.226166    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-68ptc_e17e0022-1827-408f-8f59-21dbd6d68acf/download-hf-model/0.log"
Sep 11 17:07:10.247358 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:10.247337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-b8j85_69e8fc45-2e3a-4441-871f-86e47fef596c/seaweedfs/0.log"
Sep 11 17:07:12.788069 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:12.788042    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g8n4t/perf-node-gather-daemonset-7kk6w"
Sep 11 17:07:13.841080 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:13.840995    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-7k47d_2f06dc12-13cb-4ebb-8a4e-9e894cada77b/migrator/0.log"
Sep 11 17:07:13.860420 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:13.860388    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-7k47d_2f06dc12-13cb-4ebb-8a4e-9e894cada77b/graceful-termination/0.log"
Sep 11 17:07:15.149034 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.148997    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/kube-multus-additional-cni-plugins/0.log"
Sep 11 17:07:15.165854 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.165819    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/egress-router-binary-copy/0.log"
Sep 11 17:07:15.182305 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.182285    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/cni-plugins/0.log"
Sep 11 17:07:15.200279 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.200256    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/bond-cni-plugin/0.log"
Sep 11 17:07:15.216697 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.216673    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/routeoverride-cni/0.log"
Sep 11 17:07:15.235217 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.235192    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/whereabouts-cni-bincopy/0.log"
Sep 11 17:07:15.251120 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.251099    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-68vlf_3b236a71-d94c-4de5-8fb5-3b75d3c829b3/whereabouts-cni/0.log"
Sep 11 17:07:15.550079 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.550055    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sqwzq_7b93e941-5f75-41d9-b3ec-07abf031a20d/kube-multus/0.log"
Sep 11 17:07:15.679248 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.679219    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r6hs9_90b33fa7-6adf-4900-91be-9f8c33c12eca/network-metrics-daemon/0.log"
Sep 11 17:07:15.701864 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:15.701828    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-r6hs9_90b33fa7-6adf-4900-91be-9f8c33c12eca/kube-rbac-proxy/0.log"
Sep 11 17:07:16.448764 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.448734    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/ovn-controller/0.log"
Sep 11 17:07:16.462761 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.462738    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/ovn-acl-logging/0.log"
Sep 11 17:07:16.480026 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.479996    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/kube-rbac-proxy-node/0.log"
Sep 11 17:07:16.496247 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.496223    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/kube-rbac-proxy-ovn-metrics/0.log"
Sep 11 17:07:16.512698 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.512637    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/northd/0.log"
Sep 11 17:07:16.529362 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.529337    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/nbdb/0.log"
Sep 11 17:07:16.546956 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.546925    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/sbdb/0.log"
Sep 11 17:07:16.583692 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:16.583653    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9ctcb_1aa00ea3-3996-469b-bae6-8c6b7c9fe376/ovnkube-controller/0.log"
Sep 11 17:07:17.908372 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:17.908346    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-xp82l_6348cc3f-d5db-44a6-91dc-bdc28ce89b6d/network-check-target-container/0.log"
Sep 11 17:07:18.738657 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:18.738629    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-wsddv_a373d3fc-9c59-4ea6-841b-fc50591c663e/iptables-alerter/0.log"
Sep 11 17:07:19.332284 ip-10-0-136-18 kubenswrapper[2579]: I0911 17:07:19.332257    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-k6zb9_495fbc87-c2e0-45b7-b333-4fd655f4c71c/tuned/0.log"