Sep 17 17:54:31.046988 ip-10-0-135-193 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 17:54:31.047002 ip-10-0-135-193 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 17:54:31.047012 ip-10-0-135-193 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 17:54:31.047316 ip-10-0-135-193 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 17:54:39.975930 ip-10-0-135-193 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found.
Sep 17 17:54:39.975946 ip-10-0-135-193 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot d698ea01310a4c4ba79e4043bed31626 --
Sep 17 17:56:20.250080 ip-10-0-135-193 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 17:56:23.480667 ip-10-0-135-193 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:56:23.480667 ip-10-0-135-193 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 17:56:23.480667 ip-10-0-135-193 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:56:23.480667 ip-10-0-135-193 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 17:56:23.480667 ip-10-0-135-193 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 17:56:23.481383 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.480753    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 17:56:23.485450 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485426    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.485450 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485446    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.485450 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485450    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.485450 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485453    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.485450 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485456    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485460    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485465    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485469    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485472    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485475    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485478    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485481    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485484    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485487    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485490    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485493    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485496    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485499    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485502    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485504    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485507    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485509    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485512    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.485629 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485515    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485517    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485520    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485523    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485525    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485528    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485530    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485533    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485536    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485543    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485546    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485548    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485551    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485553    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485556    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485559    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485561    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485564    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485567    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485569    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.486101 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485572    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485574    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485577    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485580    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485582    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485585    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485588    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485591    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485594    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485598    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485601    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485604    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485607    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485609    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485612    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485614    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485617    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485620    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485623    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485625    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.486589 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485628    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485631    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485634    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485636    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485639    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485642    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485644    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485648    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485651    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485653    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485656    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485659    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485664    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485668    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485671    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485674    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485676    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485679    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485681    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485684    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.487122 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485687    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485690    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485692    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485712    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485717    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485721    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485724    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485726    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485729    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485732    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485735    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485738    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485741    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485744    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485746    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.485749    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486464    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486471    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486475    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.487601 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486478    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486481    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486485    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486489    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486492    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486495    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486498    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486501    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486504    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486506    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486509    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486512    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486514    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486517    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486520    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486523    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486526    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486528    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486531    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486534    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.488073 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486537    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486539    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486542    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486544    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486547    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486549    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486552    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486554    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486557    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486559    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486562    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486564    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486567    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486570    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486573    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486575    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486586    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486589    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486592    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486595    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.488570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486597    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486600    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486603    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486606    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486609    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486613    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486617    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486622    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486625    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486628    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486630    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486633    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486636    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486638    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486641    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486643    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486646    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486649    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486651    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486654    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.489081 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486656    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486659    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486662    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486664    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486668    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486670    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486673    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486676    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486678    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486681    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486684    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486687    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486690    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486693    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486697    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486701    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486703    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486706    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486708    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.489564 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486711    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486714    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486716    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486719    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486722    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486725    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486728    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486731    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486734    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486737    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486740    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486744    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486747    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486750    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486753    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486756    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.486759    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487143    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487148    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487151    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.490031 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487154    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487157    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487160    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487163    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487166    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487169    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487171    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487174    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487177    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487179    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487182    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487184    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487187    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487190    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487193    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487196    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487198    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487201    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487205    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487208    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.490531 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487211    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487214    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487216    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487219    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487222    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487224    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487226    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487229    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487231    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487234    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487237    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487240    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487242    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487245    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487248    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487250    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487253    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487256    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487259    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487261    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.491050 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487264    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487266    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487269    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487271    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487274    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487276    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487279    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487282    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487284    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487287    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487289    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487292    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487294    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487298    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487302    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487305    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487308    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487310    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487313    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.491572 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487315    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487318    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487321    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487324    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487327    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487329    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487332    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487334    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487337    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487339    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487342    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487345    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487347    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487349    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487352    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487354    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487357    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487360    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487362    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487364    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.492037 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487367    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487370    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487373    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487376    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487378    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487381    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487383    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487386    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487389    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487392    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487394    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487398    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487401    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487403    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487407    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487411    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.487414    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487489    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487525    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487531    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487535    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 17:56:23.492512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487539    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487542    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487546    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487551    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487554    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487557    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487561    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487564    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487567    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487570    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487573    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487576    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487580    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487583    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487590    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487593    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487596    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487599    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487602    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487608    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487611    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487614    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487617    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487620    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 17:56:23.493029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487623    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487626    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487629    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487632    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487636    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487639    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487642    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487644    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487647    2566 flags.go:64] FLAG: --enable-server="true"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487650    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487654    2566 flags.go:64] FLAG: --event-burst="100"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487657    2566 flags.go:64] FLAG: --event-qps="50"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487659    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487662    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487665    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487669    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487672    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487674    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487677    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487680    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487683    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487686    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487691    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487694    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487696    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 17:56:23.493601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487699    2566 flags.go:64] FLAG: --feature-gates=""
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487703    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487709    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487712    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487715    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487718    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487721    2566 flags.go:64] FLAG: --help="false"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487724    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487727    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487730    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487733    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487737    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487740    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487743    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487746    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487748    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487751    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487754    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487757    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487760    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487763    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487766    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487769    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487772    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 17:56:23.494288 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487774    2566 flags.go:64] FLAG: --lock-file=""
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487777    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487792    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487795    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487800    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487805    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487808    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487811    2566 flags.go:64] FLAG: --logging-format="text"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487814    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487817    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487821    2566 flags.go:64] FLAG: --manifest-url=""
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487824    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487828    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487831    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487836    2566 flags.go:64] FLAG: --max-pods="110"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487838    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487842    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487844    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487847    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487850    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487853    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487856    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487863    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487866    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487869    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 17:56:23.494937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487872    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487875    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487880    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487883    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487886    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487889    2566 flags.go:64] FLAG: --port="10250"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487892    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487894    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07500d6fffb37fce3"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487898    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487901    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487904    2566 flags.go:64] FLAG: --register-node="true"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487906    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487911    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487915    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487918    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487920    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487924    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487927    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487931    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487934    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487937    2566 flags.go:64] FLAG: --runonce="false"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487940    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487943    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487946    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487949    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 17:56:23.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487951    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487954    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487957    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487960    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487963    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487966    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487968    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487971    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487974    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487977    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487979    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487985    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487987    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487990    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487994    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.487997    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488000    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488003    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488006    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488010    2566 flags.go:64] FLAG: --v="2"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488014    2566 flags.go:64] FLAG: --version="false"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488018    2566 flags.go:64] FLAG: --vmodule=""
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488022    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.488025    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488139    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.496205 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488144    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488148    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488152    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488155    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488158    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488161    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488164    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488167    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488169    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488172    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488175    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488177    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488180    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488183    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488185    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488188    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488190    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488194    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488196    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.496825 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488199    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488201    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488204    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488207    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488209    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488212    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488215    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488219    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488223    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488227    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488233    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488235    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488239    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488243    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488246    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488250    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488253    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488256    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488259    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.497285 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488261    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488264    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488267    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488269    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488272    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488275    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488277    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488280    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488283    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488285    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488288    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488291    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488293    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488296    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488298    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488301    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488304    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488306    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488309    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488311    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.497760 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488315    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488318    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488321    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488323    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488326    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488329    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488333    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488336    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488338    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488341    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488344    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488347    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488349    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488352    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488354    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488357    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488359    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488362    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488365    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.498257 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488367    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488370    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488373    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488375    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488378    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488380    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488383    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488385    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488388    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488390    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488393    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488396    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488398    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488402    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488405    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488407    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488410    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488412    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488416    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488421    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.498730 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.488424    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.489069    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496197    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496213    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496233    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496239    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496292    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496297    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496300    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496303    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496306    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496311    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496314    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496317    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.499231 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496321    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496323    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496326    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496329    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496332    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496334    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496337    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496339    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496342    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496345    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496348    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496350    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496353    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496355    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496358    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496360    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496363    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496366    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496370    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496372    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.499578 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496375    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496379    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496381    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496384    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496387    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496389    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496392    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496394    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496397    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496399    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496402    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496405    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496407    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496410    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496412    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496415    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496418    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496420    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496423    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496425    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.500085 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496428    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496430    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496433    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496435    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496438    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496441    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496443    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496446    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496448    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496452    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496454    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496457    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496460    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496462    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496466    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496469    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496471    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496474    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496476    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496479    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.500569 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496482    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496484    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496487    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496489    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496492    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496494    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496497    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496499    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496502    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496504    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496507    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496509    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496512    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496514    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496517    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496519    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496522    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496524    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496527    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.501086 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496531    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496534    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496538    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496542    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496545    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496549    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496552    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496555    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496558    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496561    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496564    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496566    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496571    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496698    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 17:56:23.501549 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496702    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496706    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496708    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496711    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496714    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496716    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496719    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496721    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496724    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496726    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496730    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496734    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496737    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496740    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496743    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496745    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496748    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496751    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496753    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 17:56:23.501901 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496756    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496760    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496763    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496767    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496770    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496772    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496775    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496789    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496793    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496795    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496798    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496801    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496803    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496805    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496809    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496811    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496814    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496816    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496819    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 17:56:23.502356 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496822    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496826    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496829    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496831    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496834    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496837    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496839    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496842    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496844    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496847    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496850    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496852    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496855    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496857    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496860    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496862    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496865    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496867    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496870    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496873    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 17:56:23.502873 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496875    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496878    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496883    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496886    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496889    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496891    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496894    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496896    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496899    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496901    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496904    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496907    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496909    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496912    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496914    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496917    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496919    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496922    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496924    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 17:56:23.503369 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496926    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496929    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496932    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496934    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496937    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496939    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496942    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496945    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496948    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496950    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496953    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496955    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496958    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496960    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496963    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496965    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496968    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496970    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496973    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496975    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 17:56:23.503826 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:23.496978    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.496982    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.497079    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.499719    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.500536    2566 server.go:1019] "Starting client certificate rotation"
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.500660    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.500689    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 17:56:23.504634 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.502830    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:56:23.506383 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.506340    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 17:56:23.512334 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.512316    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 17 17:56:23.513904 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.513890    2566 log.go:25] "Validated CRI v1 image API"
Sep 17 17:56:23.514635 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.514621    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 17:56:23.517351 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.517323    2566 fs.go:135] Filesystem UUIDs: map[0eff25ac-a38c-43e9-a2c2-1b150aabdb01:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8ae6256d-1905-4360-b202-b349d31184e7:/dev/nvme0n1p4]
Sep 17 17:56:23.517412 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.517350    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 17:56:23.521501 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.521399    2566 manager.go:216] Machine: {Timestamp:2026-09-17 17:56:23.520920079 +0000 UTC m=+1.944103965 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3106320 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c1b8e2d20879aa211795c3fbfd065 SystemUUID:ec2c1b8e-2d20-879a-a211-795c3fbfd065 BootID:d698ea01-310a-4c4b-a79e-4043bed31626 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f5:f0:3c:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f5:f0:3c:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ee:39:fc:2c:d7:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 17:56:23.521501 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.521498    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 17:56:23.521599 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.521568    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 17:56:23.523206 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523185    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 17:56:23.523343 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523209    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-193.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 17:56:23.523890 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523880    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 17:56:23.523921 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523893    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 17:56:23.523921 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523900    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:56:23.523921 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.523912    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 17:56:23.525340 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.525330    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 17:56:23.529363 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529352    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:56:23.529470 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529460    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 17:56:23.529499 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529491    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 17:56:23.529531 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529501    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 17:56:23.529531 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529512    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 17:56:23.529531 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529523    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 17 17:56:23.529531 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529531    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 17:56:23.529929 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529916    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:56:23.529972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.529950    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 17:56:23.534922 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.534899    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 17:56:23.535021 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.534990    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:56:23.538685 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.538666    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 17:56:23.538748 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.538699    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 17:56:23.539924 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539912    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 17:56:23.539972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539929    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 17:56:23.539972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539936    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 17:56:23.539972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539943    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 17:56:23.539972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539957    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 17:56:23.539972 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539965    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539979    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539985    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539992    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.539997    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.540005    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.540011    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.540034    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 17:56:23.540115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.540039    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 17:56:23.543748 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.543723    2566 server.go:1268] "Started kubelet"
Sep 17 17:56:23.543868 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.543822    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 17:56:23.543937 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.543825    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 17:56:23.543986 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.543960    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 17:56:23.544641 ip-10-0-135-193 systemd[1]: Started Kubernetes Kubelet.
Sep 17 17:56:23.545225 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.545037    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 17:56:23.545225 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.545205    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 17:56:23.545979 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.545957    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-193.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 17:56:23.546046 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.546005    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 17:56:23.546217 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.546193    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 17:56:23.548797 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.548767    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 17:56:23.552416 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.552398    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 17:56:23.552416 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.552407    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 17:56:23.552539 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.552476    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:23.552979 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.552963    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 17:56:23.552979 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.552964    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 17:56:23.552979 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.552983    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 17:56:23.553115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.553064    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 17:56:23.553115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.553072    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 17:56:23.553907 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.553890    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 17:56:23.553998 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.553933    2566 factory.go:55] Registering systemd factory
Sep 17 17:56:23.553998 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.553946    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 17 17:56:23.554190 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.554174    2566 factory.go:153] Registering CRI-O factory
Sep 17 17:56:23.554260 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.554218    2566 factory.go:223] Registration of the crio container factory successfully
Sep 17 17:56:23.554260 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.554236    2566 factory.go:103] Registering Raw factory
Sep 17 17:56:23.554260 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.554249    2566 manager.go:1196] Started watching for new ooms in manager
Sep 17 17:56:23.554442 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.554420    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 17:56:23.554518 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.554493    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-193.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 17:56:23.554675 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.554656    2566 manager.go:349] Starting recovery of all containers
Sep 17 17:56:23.572939 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.572918    2566 manager.go:354] Recovery completed
Sep 17 17:56:23.579306 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.579291    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.580935 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.580914    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.581006 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.580945    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.581006 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.580955    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.581489 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.581477    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 17:56:23.581489 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.581488    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 17:56:23.581584 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.581492    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.581584 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.581504    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 17:56:23.582144 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.582130    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.582207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.582155    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.582207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.582164    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.583693 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.583681    2566 policy_none.go:49] "None policy: Start"
Sep 17 17:56:23.583737 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.583697    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 17:56:23.583737 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.583707    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 17:56:23.585677 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.585662    2566 policy_none.go:47] "Start"
Sep 17 17:56:23.589866 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.588806    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18d62d5df8151939  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-09-17 17:56:23.543691577 +0000 UTC m=+1.966875472,LastTimestamp:2026-09-17 17:56:23.543691577 +0000 UTC m=+1.966875472,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Sep 17 17:56:23.594355 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.594296    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18d62d5dfa4d5b45  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-193.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-09-17 17:56:23.580932933 +0000 UTC m=+2.004116819,LastTimestamp:2026-09-17 17:56:23.580932933 +0000 UTC m=+2.004116819,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Sep 17 17:56:23.597842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.597814    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fxwtb"
Sep 17 17:56:23.599367 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.599293    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18d62d5dfa4d9b6c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-193.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-09-17 17:56:23.580949356 +0000 UTC m=+2.004133243,LastTimestamp:2026-09-17 17:56:23.580949356 +0000 UTC m=+2.004133243,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Sep 17 17:56:23.604774 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.604756    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fxwtb"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.607500    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-193.ec2.internal.18d62d5dfa4dbf15  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-193.ec2.internal,UID:ip-10-0-135-193.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-193.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-193.ec2.internal,},FirstTimestamp:2026-09-17 17:56:23.580958485 +0000 UTC m=+2.004142371,LastTimestamp:2026-09-17 17:56:23.580958485 +0000 UTC m=+2.004142371,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-193.ec2.internal,}"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619434    2566 manager.go:337] "Starting Device Plugin manager"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.619471    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619480    2566 server.go:87] "Starting device plugin registration server"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619712    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619735    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619819    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619911    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.619919    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.620495    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 17:56:23.628283 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.620532    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:23.645894 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.645873    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 17:56:23.647069 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.647050    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 17:56:23.647151 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.647089    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 17:56:23.647151 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.647111    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 17:56:23.647248 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.647155    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 17:56:23.649410 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.649391    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:23.720125 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.720101    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.720995 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.720980    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.721082 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.721009    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.721082 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.721022    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.721082 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.721045    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.730598 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.730580    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.730691 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.730601    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-193.ec2.internal\": node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:23.747594 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.747569    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"]
Sep 17 17:56:23.747661 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.747630    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.748422 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.748394    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.748422 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.748420    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.748519 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.748429    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.750745 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.750733    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.750887 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.750873    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.750930 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.750901    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.751505 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751490    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.751586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751512    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.751586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751522    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.751586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751556    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.751586 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751582    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.751793 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.751598    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.753528 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.753509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.753608 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.753532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.753608 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.753550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46e5674716c4f3612e5b47f16efafbea-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"46e5674716c4f3612e5b47f16efafbea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.753775 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.753762    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.753830 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.753799    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:23.754458 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.754441    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:23.754517 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.754475    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:23.754517 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.754493    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:23.772314 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.772297    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-193.ec2.internal\" not found" node="ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.775994 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.775980    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-193.ec2.internal\" not found" node="ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.781352 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.781328    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:23.854095 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854067    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.854095 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.854235 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.854235 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854134    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46e5674716c4f3612e5b47f16efafbea-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"46e5674716c4f3612e5b47f16efafbea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.854235 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854161    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46e5674716c4f3612e5b47f16efafbea-config\") pod \"kube-apiserver-proxy-ip-10-0-135-193.ec2.internal\" (UID: \"46e5674716c4f3612e5b47f16efafbea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.854235 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:23.854191    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4fdc744cec5cb7b71204632f166ba7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal\" (UID: \"cb4fdc744cec5cb7b71204632f166ba7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:23.882392 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.882374    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:23.983434 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:23.983388    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.076190 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.076150    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:24.080201 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.080177    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:24.083827 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.083809    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.184606 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.184569    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.285187 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.285115    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.376979 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.376949    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:24.385680 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.385661    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.486192 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.486162    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.500326 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.500309    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 17:56:24.500451 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.500432    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 17:56:24.500489 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.500448    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 17:56:24.586451 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.586426    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.594591 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.594565    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 17:56:24.595604 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.595586    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 17:56:24.595696 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.595622    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 17:56:24.595696 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.595637    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeHasSufficientPID"
Sep 17 17:56:24.611721 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.611694    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 17:51:23 +0000 UTC" deadline="2028-04-06 09:18:36.210179389 +0000 UTC"
Sep 17 17:56:24.611721 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.611719    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13599h22m11.598463298s"
Sep 17 17:56:24.674461 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:24.674436    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4fdc744cec5cb7b71204632f166ba7.slice/crio-64ab31357af92602247d6578046b04f9aae04712aeb035ff9f2f4c1633283632 WatchSource:0}: Error finding container 64ab31357af92602247d6578046b04f9aae04712aeb035ff9f2f4c1633283632: Status 404 returned error can't find the container with id 64ab31357af92602247d6578046b04f9aae04712aeb035ff9f2f4c1633283632
Sep 17 17:56:24.674938 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:24.674921    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e5674716c4f3612e5b47f16efafbea.slice/crio-6be2cf0431327d288ab9fad08276dcd591bf48a33da67d90e044e86808ca9f35 WatchSource:0}: Error finding container 6be2cf0431327d288ab9fad08276dcd591bf48a33da67d90e044e86808ca9f35: Status 404 returned error can't find the container with id 6be2cf0431327d288ab9fad08276dcd591bf48a33da67d90e044e86808ca9f35
Sep 17 17:56:24.680921 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.680908    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 17:56:24.686649 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.686629    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.787476 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.787439    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.871108 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.871035    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:24.887577 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:24.887550    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-193.ec2.internal\" not found"
Sep 17 17:56:24.932236 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.932208    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:24.953622 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.953605    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:24.968660 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.968644    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:56:24.969507 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.969495    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal"
Sep 17 17:56:24.980382 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:24.980369    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 17:56:25.531478 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.531449    2566 apiserver.go:52] "Watching apiserver"
Sep 17 17:56:25.540551 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.540531    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 17:56:25.541031 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.541008    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-qfd6w","kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq","openshift-cluster-node-tuning-operator/tuned-c7zh2","openshift-dns/node-resolver-7s5jk","openshift-image-registry/node-ca-prs4c","openshift-multus/network-metrics-daemon-8bqps","openshift-network-operator/iptables-alerter-qdf8r","kube-system/konnectivity-agent-wg4sr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal","openshift-multus/multus-additional-cni-plugins-rtjw7","openshift-multus/multus-bwbk5","openshift-network-diagnostics/network-check-target-6zqk5","openshift-ovn-kubernetes/ovnkube-node-dspj8"]
Sep 17 17:56:25.543564 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.543541    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.546260 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.546234    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 17:56:25.546365 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.546324    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.546436 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.546419    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-grfdw\""
Sep 17 17:56:25.546572 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.546550    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.546828 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.546808    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.548996 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.548977    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.551379 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.551363    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.551755 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.551736    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 17:56:25.552006 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.551989    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.552132 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.552110    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qh27t\""
Sep 17 17:56:25.552199 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.552164    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 17:56:25.552285 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.552271    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553283    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553291    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553322    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553350    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hdlzw\""
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553441    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 17:56:25.553500 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553471    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 17:56:25.553880 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553532    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 17:56:25.553880 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.553747    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.554362 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.554343    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.554507 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.554489    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.554978 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.554959    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gzwx7\""
Sep 17 17:56:25.555688 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.555674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.558119 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.558104    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.558660 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.558632    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.558759 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.558730    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:25.560872 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.560854    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.560956 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.560908    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 17:56:25.560956 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.560914    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:25.561117 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.561099    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bjnpk\""
Sep 17 17:56:25.561170 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.561118    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 17:56:25.561214 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.561190    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cpt87\""
Sep 17 17:56:25.563682 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-host\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.563771 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563694    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-modprobe-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.563771 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-kubernetes\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.563771 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563753    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-var-lib-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563777    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-node-log\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563842    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-systemd\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563864    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-systemd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563888    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563910    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-config\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.563952 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxws7\" (UniqueName: \"kubernetes.io/projected/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-kube-api-access-sxws7\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.563995    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-sys-fs\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4sr4\" (UniqueName: \"kubernetes.io/projected/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kube-api-access-k4sr4\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-tuned\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564081    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-systemd-units\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564083    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.564241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-netd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564541 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564444    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovn-node-metrics-cert\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564541 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564494    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-script-lib\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564541 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564532    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-socket-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564563    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-registration-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-sys\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564647    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/855b4e5c-fbda-4f66-accd-a5dbac52c006-iptables-alerter-script\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-bin\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1515c65-4fb6-4f0a-9e62-649e22abdbff-tmp-dir\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564734    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrl6q\" (UniqueName: \"kubernetes.io/projected/855b4e5c-fbda-4f66-accd-a5dbac52c006-kube-api-access-rrl6q\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564757    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-kubelet\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.564818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-run\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564868    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-device-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-var-lib-kubelet\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564922    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysconfig\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.564950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-conf\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565207 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.565002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-tmp\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.565492 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.565305    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1515c65-4fb6-4f0a-9e62-649e22abdbff-hosts-file\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.565656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-log-socket\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.565703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwzq6\" (UniqueName: \"kubernetes.io/projected/5c0b01b6-4b66-4b41-873e-e8be72a209e8-kube-api-access-jwzq6\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566168    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkbv5\" (UniqueName: \"kubernetes.io/projected/a1515c65-4fb6-4f0a-9e62-649e22abdbff-kube-api-access-vkbv5\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-slash\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566373    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-netns\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566415    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-etc-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566602    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-ovn\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566640    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-env-overrides\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.566842 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566764    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/855b4e5c-fbda-4f66-accd-a5dbac52c006-host-slash\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.567317 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.566849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-lib-modules\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.567371 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.567321    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.568713 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.568689    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-znm7s\""
Sep 17 17:56:25.569313 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.569253    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.569981 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.569957    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 17:56:25.570138 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.570121    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.570814 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.570750    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.572658 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.572552    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 17:56:25.572909 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.572893    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.574519 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.574498    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 17:56:25.574626 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.574529    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 17:56:25.574978 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.574957    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:25.575057 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.575019    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:25.575175 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.575141    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 17:56:25.575362 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.575247    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zs2vt\""
Sep 17 17:56:25.576364 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.575886    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 17:56:25.576364 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.576027    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-97qm6\""
Sep 17 17:56:25.576364 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.576096    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 17:56:25.576364 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.576235    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 17:56:25.610565 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.610540    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jglh"
Sep 17 17:56:25.620555 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.620533    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jglh"
Sep 17 17:56:25.651986 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.651939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" event={"ID":"46e5674716c4f3612e5b47f16efafbea","Type":"ContainerStarted","Data":"6be2cf0431327d288ab9fad08276dcd591bf48a33da67d90e044e86808ca9f35"}
Sep 17 17:56:25.653122 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.653097    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"cb4fdc744cec5cb7b71204632f166ba7","Type":"ContainerStarted","Data":"64ab31357af92602247d6578046b04f9aae04712aeb035ff9f2f4c1633283632"}
Sep 17 17:56:25.653974 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.653952    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 17:56:25.667085 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-netd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667085 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovn-node-metrics-cert\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-script-lib\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667134    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-registration-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-netd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667183    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-dbus\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/855b4e5c-fbda-4f66-accd-a5dbac52c006-iptables-alerter-script\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667223    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-registration-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667233    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-bin\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667268 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-kubelet\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667282    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9062e02-a63c-4406-9782-5c0095128a9e-host\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667325    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-kubelet\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667348    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-multus-certs\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-cni-bin\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kbnj\" (UniqueName: \"kubernetes.io/projected/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-kube-api-access-5kbnj\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jkzb\" (UniqueName: \"kubernetes.io/projected/e9062e02-a63c-4406-9782-5c0095128a9e-kube-api-access-5jkzb\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667416    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667444    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-conf\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667467    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-system-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667484    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-hostroot\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-system-cni-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667548    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-kubelet\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.667732 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667732    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667814    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cni-binary-copy\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667840    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkbv5\" (UniqueName: \"kubernetes.io/projected/a1515c65-4fb6-4f0a-9e62-649e22abdbff-kube-api-access-vkbv5\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-slash\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667867    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-script-lib\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-netns\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667871    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysctl-conf\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667914    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-ovn\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667954    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-slash\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667957    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-netns\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.667976    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-env-overrides\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-ovn\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668012    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-etc-kubernetes\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668039    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/855b4e5c-fbda-4f66-accd-a5dbac52c006-iptables-alerter-script\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668058    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxws7\" (UniqueName: \"kubernetes.io/projected/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-kube-api-access-sxws7\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.668554 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/855b4e5c-fbda-4f66-accd-a5dbac52c006-host-slash\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668137    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-lib-modules\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668160    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-host\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668184    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-var-lib-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668212    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-node-log\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668240    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-multus\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-conf-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668295    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-systemd\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-config\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668393    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-node-log\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668422    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-k8s-cni-cncf-io\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-bin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-cnibin\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668529    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-lib-modules\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668543    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb5m2\" (UniqueName: \"kubernetes.io/projected/9cad02b4-c802-4273-b90c-55f9ca072e76-kube-api-access-qb5m2\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.669454 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668577    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-var-lib-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668574    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-tuned\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-run-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668628    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-systemd-units\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/855b4e5c-fbda-4f66-accd-a5dbac52c006-host-slash\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-systemd\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668650    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-socket-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668672    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-host\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668677    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6cwv\" (UniqueName: \"kubernetes.io/projected/97e76c9b-8f82-49a0-9714-fded16b71ee7-kube-api-access-q6cwv\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668710    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-sys\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668737    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668741    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-systemd-units\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668805    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-sys\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668816    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-netns\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668843    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8dfb0243-cd9b-4843-b14a-4bde217501c8-agent-certs\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1515c65-4fb6-4f0a-9e62-649e22abdbff-tmp-dir\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668908    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrl6q\" (UniqueName: \"kubernetes.io/projected/855b4e5c-fbda-4f66-accd-a5dbac52c006-kube-api-access-rrl6q\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.670284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668923    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-socket-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668945    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-env-overrides\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668961    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-kubelet-config\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.668994    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-run\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669020    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-device-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669047    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-socket-dir-parent\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669072    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-daemon-config\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669075    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-run\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669095    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-var-lib-kubelet\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-device-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysconfig\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-tmp\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669184    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1515c65-4fb6-4f0a-9e62-649e22abdbff-tmp-dir\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669194    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1515c65-4fb6-4f0a-9e62-649e22abdbff-hosts-file\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-sysconfig\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671153 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669204    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-var-lib-kubelet\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669219    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-log-socket\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669254    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-log-socket\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1515c65-4fb6-4f0a-9e62-649e22abdbff-hosts-file\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669265    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669303    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669304    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cnibin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669335    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8dfb0243-cd9b-4843-b14a-4bde217501c8-konnectivity-ca\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669358    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwzq6\" (UniqueName: \"kubernetes.io/projected/5c0b01b6-4b66-4b41-873e-e8be72a209e8-kube-api-access-jwzq6\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-etc-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669404    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669428    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4sr4\" (UniqueName: \"kubernetes.io/projected/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kube-api-access-k4sr4\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-etc-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-modprobe-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-kubernetes\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.671901 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669545    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-systemd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-sys-fs\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9062e02-a63c-4406-9782-5c0095128a9e-serviceca\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-kubernetes\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669696    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-sys-fs\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669705    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-openvswitch\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669802    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovnkube-config\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669804    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-run-systemd\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.669883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-modprobe-d\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.671747    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-tmp\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.672505    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5c0b01b6-4b66-4b41-873e-e8be72a209e8-etc-tuned\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.672631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.672625    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-ovn-node-metrics-cert\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.679715 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.679691    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkbv5\" (UniqueName: \"kubernetes.io/projected/a1515c65-4fb6-4f0a-9e62-649e22abdbff-kube-api-access-vkbv5\") pod \"node-resolver-7s5jk\" (UID: \"a1515c65-4fb6-4f0a-9e62-649e22abdbff\") " pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.680948 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.680920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxws7\" (UniqueName: \"kubernetes.io/projected/8989de48-e3c3-40e1-b0cb-1db2ea9b570d-kube-api-access-sxws7\") pod \"ovnkube-node-dspj8\" (UID: \"8989de48-e3c3-40e1-b0cb-1db2ea9b570d\") " pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.682885 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.682864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwzq6\" (UniqueName: \"kubernetes.io/projected/5c0b01b6-4b66-4b41-873e-e8be72a209e8-kube-api-access-jwzq6\") pod \"tuned-c7zh2\" (UID: \"5c0b01b6-4b66-4b41-873e-e8be72a209e8\") " pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.683487 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.683441    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrl6q\" (UniqueName: \"kubernetes.io/projected/855b4e5c-fbda-4f66-accd-a5dbac52c006-kube-api-access-rrl6q\") pod \"iptables-alerter-qdf8r\" (UID: \"855b4e5c-fbda-4f66-accd-a5dbac52c006\") " pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.683583 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.683515    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:25.684763 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.684742    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4sr4\" (UniqueName: \"kubernetes.io/projected/cc01f23a-d3ca-4f95-bbc5-e39abe35f39c-kube-api-access-k4sr4\") pod \"aws-ebs-csi-driver-node-xszlq\" (UID: \"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.770651 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cnibin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770770 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8dfb0243-cd9b-4843-b14a-4bde217501c8-konnectivity-ca\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.770770 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.770770 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770717    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9062e02-a63c-4406-9782-5c0095128a9e-serviceca\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.770770 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-dbus\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.770770 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9062e02-a63c-4406-9782-5c0095128a9e-host\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770810    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-kubelet\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770833    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-multus-certs\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kbnj\" (UniqueName: \"kubernetes.io/projected/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-kube-api-access-5kbnj\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770875    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jkzb\" (UniqueName: \"kubernetes.io/projected/e9062e02-a63c-4406-9782-5c0095128a9e-kube-api-access-5jkzb\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-system-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-hostroot\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.770985 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770964    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-system-cni-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.770988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771011    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cni-binary-copy\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771040    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-etc-kubernetes\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771063    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-multus\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771120    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-conf-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771146    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-k8s-cni-cncf-io\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-bin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771192    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:25.771237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771216    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-cnibin\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771237    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e9062e02-a63c-4406-9782-5c0095128a9e-serviceca\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8dfb0243-cd9b-4843-b14a-4bde217501c8-konnectivity-ca\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e9062e02-a63c-4406-9782-5c0095128a9e-host\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771325    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cnibin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb5m2\" (UniqueName: \"kubernetes.io/projected/9cad02b4-c802-4273-b90c-55f9ca072e76-kube-api-access-qb5m2\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771342    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-kubelet\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771343    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-binary-copy\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771373    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6cwv\" (UniqueName: \"kubernetes.io/projected/97e76c9b-8f82-49a0-9714-fded16b71ee7-kube-api-access-q6cwv\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771395    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-multus-certs\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771400    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771424    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-netns\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771446    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8dfb0243-cd9b-4843-b14a-4bde217501c8-agent-certs\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771512 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771497    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-kubelet-config\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-socket-dir-parent\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771525    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-etc-kubernetes\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771556    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-dbus\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771544    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-daemon-config\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771608    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-k8s-cni-cncf-io\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-cnibin\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771676    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-conf-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771717    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-run-netns\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771738    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-socket-dir-parent\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771771    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-system-cni-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771775    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-bin\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771828    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-system-cni-dir\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.771854    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-hostroot\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.771894    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9cad02b4-c802-4273-b90c-55f9ca072e76-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.771915 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.771915    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:26.271896185 +0000 UTC m=+4.695080059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.771953    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-host-var-lib-cni-multus\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.772196    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:26.272178332 +0000 UTC m=+4.695362206 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772260    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0b23d48e-e957-4278-b558-443a13430040-kubelet-config\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772299    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-cni-binary-copy\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-multus-daemon-config\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.772526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772461    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.772875 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.772718    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9cad02b4-c802-4273-b90c-55f9ca072e76-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.774642 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.774616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8dfb0243-cd9b-4843-b14a-4bde217501c8-agent-certs\") pod \"konnectivity-agent-wg4sr\" (UID: \"8dfb0243-cd9b-4843-b14a-4bde217501c8\") " pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.784171 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.784117    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:25.784171 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.784140    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:25.784171 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.784150    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:25.784378 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:25.784205    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:26.28418742 +0000 UTC m=+4.707371299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:25.786690 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.786648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb5m2\" (UniqueName: \"kubernetes.io/projected/9cad02b4-c802-4273-b90c-55f9ca072e76-kube-api-access-qb5m2\") pod \"multus-additional-cni-plugins-rtjw7\" (UID: \"9cad02b4-c802-4273-b90c-55f9ca072e76\") " pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.786690 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.786685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jkzb\" (UniqueName: \"kubernetes.io/projected/e9062e02-a63c-4406-9782-5c0095128a9e-kube-api-access-5jkzb\") pod \"node-ca-prs4c\" (UID: \"e9062e02-a63c-4406-9782-5c0095128a9e\") " pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.787082 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.787061    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6cwv\" (UniqueName: \"kubernetes.io/projected/97e76c9b-8f82-49a0-9714-fded16b71ee7-kube-api-access-q6cwv\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:25.787158 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.787063    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kbnj\" (UniqueName: \"kubernetes.io/projected/cd2d6bef-a10f-42e0-b02e-b73f3510a19b-kube-api-access-5kbnj\") pod \"multus-bwbk5\" (UID: \"cd2d6bef-a10f-42e0-b02e-b73f3510a19b\") " pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:25.856099 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.856068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qdf8r"
Sep 17 17:56:25.861811 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.861775    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:25.871602 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.871575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq"
Sep 17 17:56:25.877526 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.877505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c7zh2"
Sep 17 17:56:25.884219 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.884198    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7s5jk"
Sep 17 17:56:25.892034 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.892012    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:25.900622 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.900603    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-prs4c"
Sep 17 17:56:25.906616 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.906596    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rtjw7"
Sep 17 17:56:25.912092 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:25.912073    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bwbk5"
Sep 17 17:56:26.275458 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.275431    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:26.275617 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.275476    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:26.275617 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.275589    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:26.275713 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.275656    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:27.275640346 +0000 UTC m=+5.698824242 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:26.275713 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.275594    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:26.275813 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.275720    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:27.275706179 +0000 UTC m=+5.698890052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:26.376764 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.376731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:26.376946 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.376910    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:26.376946 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.376933    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:26.376946 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.376945    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:26.377103 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:26.377015    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:27.376994142 +0000 UTC m=+5.800178029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:26.603570 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.603540    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cad02b4_c802_4273_b90c_55f9ca072e76.slice/crio-23b28bdd6c1378fb06784f80d0cd934627ea7c76f92ad8fcbc8ddeb8bc3d3cfd WatchSource:0}: Error finding container 23b28bdd6c1378fb06784f80d0cd934627ea7c76f92ad8fcbc8ddeb8bc3d3cfd: Status 404 returned error can't find the container with id 23b28bdd6c1378fb06784f80d0cd934627ea7c76f92ad8fcbc8ddeb8bc3d3cfd
Sep 17 17:56:26.605683 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.605657    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1515c65_4fb6_4f0a_9e62_649e22abdbff.slice/crio-9c0dcd7202b78aceab93fa39202dbedc2bef3dc2f1b27a9956bc3b1ac2c65cf8 WatchSource:0}: Error finding container 9c0dcd7202b78aceab93fa39202dbedc2bef3dc2f1b27a9956bc3b1ac2c65cf8: Status 404 returned error can't find the container with id 9c0dcd7202b78aceab93fa39202dbedc2bef3dc2f1b27a9956bc3b1ac2c65cf8
Sep 17 17:56:26.608661 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.608638    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8989de48_e3c3_40e1_b0cb_1db2ea9b570d.slice/crio-4015d9d477545c9bfaa5f90c3505ea40e15c7f55a713978981ed66be70b07741 WatchSource:0}: Error finding container 4015d9d477545c9bfaa5f90c3505ea40e15c7f55a713978981ed66be70b07741: Status 404 returned error can't find the container with id 4015d9d477545c9bfaa5f90c3505ea40e15c7f55a713978981ed66be70b07741
Sep 17 17:56:26.612666 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.612640    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc01f23a_d3ca_4f95_bbc5_e39abe35f39c.slice/crio-09ad7d297e05d7df417cb3f3ee2ad3a2960c4bfd7147d2d45e31be4799404efa WatchSource:0}: Error finding container 09ad7d297e05d7df417cb3f3ee2ad3a2960c4bfd7147d2d45e31be4799404efa: Status 404 returned error can't find the container with id 09ad7d297e05d7df417cb3f3ee2ad3a2960c4bfd7147d2d45e31be4799404efa
Sep 17 17:56:26.613829 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.613667    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd2d6bef_a10f_42e0_b02e_b73f3510a19b.slice/crio-beda338a753398eb13ac09fa8e5152deb025220b72d3cac068cac9d25888ee7d WatchSource:0}: Error finding container beda338a753398eb13ac09fa8e5152deb025220b72d3cac068cac9d25888ee7d: Status 404 returned error can't find the container with id beda338a753398eb13ac09fa8e5152deb025220b72d3cac068cac9d25888ee7d
Sep 17 17:56:26.614827 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.614715    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dfb0243_cd9b_4843_b14a_4bde217501c8.slice/crio-6359e43ebecd69d1700f491b0a8f3e8d5cb967f925fcec7535d170626be70848 WatchSource:0}: Error finding container 6359e43ebecd69d1700f491b0a8f3e8d5cb967f925fcec7535d170626be70848: Status 404 returned error can't find the container with id 6359e43ebecd69d1700f491b0a8f3e8d5cb967f925fcec7535d170626be70848
Sep 17 17:56:26.615420 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.615391    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855b4e5c_fbda_4f66_accd_a5dbac52c006.slice/crio-607b9bdcbde2417008915a9befee6da96c476190f8955273f22ef1bd6131baf5 WatchSource:0}: Error finding container 607b9bdcbde2417008915a9befee6da96c476190f8955273f22ef1bd6131baf5: Status 404 returned error can't find the container with id 607b9bdcbde2417008915a9befee6da96c476190f8955273f22ef1bd6131baf5
Sep 17 17:56:26.617329 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:26.616970    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9062e02_a63c_4406_9782_5c0095128a9e.slice/crio-794909311cd8c2bf30361c28efdc699106acf7c6253ff5b95c9bd96d579cda05 WatchSource:0}: Error finding container 794909311cd8c2bf30361c28efdc699106acf7c6253ff5b95c9bd96d579cda05: Status 404 returned error can't find the container with id 794909311cd8c2bf30361c28efdc699106acf7c6253ff5b95c9bd96d579cda05
Sep 17 17:56:26.624322 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.624299    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:51:25 +0000 UTC" deadline="2028-02-26 12:58:42.640020079 +0000 UTC"
Sep 17 17:56:26.624376 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.624323    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12643h2m16.015700999s"
Sep 17 17:56:26.656052 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.656026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c7zh2" event={"ID":"5c0b01b6-4b66-4b41-873e-e8be72a209e8","Type":"ContainerStarted","Data":"1b57c4cec0e941fd4a6ef32089e7739939645c3d01001ded1537d47e1ad500ab"}
Sep 17 17:56:26.657058 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.657037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-prs4c" event={"ID":"e9062e02-a63c-4406-9782-5c0095128a9e","Type":"ContainerStarted","Data":"794909311cd8c2bf30361c28efdc699106acf7c6253ff5b95c9bd96d579cda05"}
Sep 17 17:56:26.661390 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.661290    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdf8r" event={"ID":"855b4e5c-fbda-4f66-accd-a5dbac52c006","Type":"ContainerStarted","Data":"607b9bdcbde2417008915a9befee6da96c476190f8955273f22ef1bd6131baf5"}
Sep 17 17:56:26.663945 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.663922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwbk5" event={"ID":"cd2d6bef-a10f-42e0-b02e-b73f3510a19b","Type":"ContainerStarted","Data":"beda338a753398eb13ac09fa8e5152deb025220b72d3cac068cac9d25888ee7d"}
Sep 17 17:56:26.665034 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.665012    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"4015d9d477545c9bfaa5f90c3505ea40e15c7f55a713978981ed66be70b07741"}
Sep 17 17:56:26.669028 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.669005    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7s5jk" event={"ID":"a1515c65-4fb6-4f0a-9e62-649e22abdbff","Type":"ContainerStarted","Data":"9c0dcd7202b78aceab93fa39202dbedc2bef3dc2f1b27a9956bc3b1ac2c65cf8"}
Sep 17 17:56:26.670087 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.670067    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerStarted","Data":"23b28bdd6c1378fb06784f80d0cd934627ea7c76f92ad8fcbc8ddeb8bc3d3cfd"}
Sep 17 17:56:26.671221 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.671202    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg4sr" event={"ID":"8dfb0243-cd9b-4843-b14a-4bde217501c8","Type":"ContainerStarted","Data":"6359e43ebecd69d1700f491b0a8f3e8d5cb967f925fcec7535d170626be70848"}
Sep 17 17:56:26.672356 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:26.672334    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq" event={"ID":"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c","Type":"ContainerStarted","Data":"09ad7d297e05d7df417cb3f3ee2ad3a2960c4bfd7147d2d45e31be4799404efa"}
Sep 17 17:56:27.286367 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.286027    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:27.286367 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.286348    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:27.286597 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.286472    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:27.286597 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.286532    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:29.286513695 +0000 UTC m=+7.709697582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:27.286969 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.286945    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:27.287069 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.287002    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:29.286987564 +0000 UTC m=+7.710171451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:27.387836 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.387804    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:27.388037 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.388018    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:27.388111 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.388044    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:27.388111 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.388056    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:27.388222 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.388136    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:29.388116814 +0000 UTC m=+7.811300908 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:27.390154 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.390131    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 17:56:27.624810 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.624698    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 17:51:25 +0000 UTC" deadline="2028-06-20 11:36:51.818806232 +0000 UTC"
Sep 17 17:56:27.624810 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.624755    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15401h40m24.194054523s"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.650323    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.650443    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.650531    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.650626    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.650681    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:27.651174 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:27.650744    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:27.683055 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.683020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" event={"ID":"46e5674716c4f3612e5b47f16efafbea","Type":"ContainerStarted","Data":"d265d7be9c01908d13317703eb3f5eb6a45975095605f7fcf97874f5f393ea15"}
Sep 17 17:56:27.687855 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.687827    2566 generic.go:356] "Generic (PLEG): container finished" podID="cb4fdc744cec5cb7b71204632f166ba7" containerID="0a88343a620b788e6f41e4de63c92d1ef69e7ece5da406cbb82304aca5ac0d5c" exitCode=0
Sep 17 17:56:27.687965 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.687871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"cb4fdc744cec5cb7b71204632f166ba7","Type":"ContainerDied","Data":"0a88343a620b788e6f41e4de63c92d1ef69e7ece5da406cbb82304aca5ac0d5c"}
Sep 17 17:56:27.722350 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:27.722297    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-193.ec2.internal" podStartSLOduration=3.722276218 podStartE2EDuration="3.722276218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:27.701465591 +0000 UTC m=+6.124649489" watchObservedRunningTime="2026-09-17 17:56:27.722276218 +0000 UTC m=+6.145460114"
Sep 17 17:56:28.712259 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:28.712180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" event={"ID":"cb4fdc744cec5cb7b71204632f166ba7","Type":"ContainerStarted","Data":"c76130d4b3d42d46c3f9ffbadf23cf2a9e78f6145585a231174a8ebb3dd48856"}
Sep 17 17:56:29.304266 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.304229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:29.304453 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.304298    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:29.304453 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.304447    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:29.304571 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.304506    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:33.304488568 +0000 UTC m=+11.727672445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:29.305040 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.305019    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:29.305143 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.305078    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:33.305062752 +0000 UTC m=+11.728246643 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:29.405701 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.405665    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:29.405872 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.405848    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:29.405872 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.405869    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:29.405966 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.405880    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:29.405966 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.405936    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:33.405918706 +0000 UTC m=+11.829102584 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.650528    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.650699    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.650741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:29.650768    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.650860    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:29.651256 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:29.650932    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:31.648190 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:31.648154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:31.648689 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:31.648286    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:31.648689 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:31.648328    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:31.648689 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:31.648418    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:31.648689 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:31.648467    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:31.648689 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:31.648531    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:33.337058 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.337018    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:33.337529 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.337086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:33.337529 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.337180    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:33.337529 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.337220    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:33.337529 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.337264    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:41.337241239 +0000 UTC m=+19.760425120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:33.337529 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.337288    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:41.337276336 +0000 UTC m=+19.760460215 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:33.438345 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.438308    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:33.438531 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.438499    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:33.438531 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.438526    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:33.438649 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.438546    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:33.438649 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.438614    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:41.438594661 +0000 UTC m=+19.861778538 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:33.648542 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.648383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:33.648542 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.648481    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:33.648542 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:33.648510    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:33.648820 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.648593    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:33.648875 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.648808    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:33.648875 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:33.648833    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:35.648560 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:35.648524    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:35.649022 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:35.648653    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:35.649022 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:35.648524    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:35.649022 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:35.648809    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:35.649022 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:35.648874    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:35.649022 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:35.648943    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:37.648018 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:37.647987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:37.648558 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:37.647987    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:37.648558 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:37.648106    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:37.648558 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:37.648116    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:37.648558 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:37.648188    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:37.648558 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:37.648254    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:39.647487 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:39.647399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:39.647487 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:39.647433    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:39.647487 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:39.647399    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:39.647977 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:39.647516    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:39.647977 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:39.647636    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:39.647977 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:39.647724    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:41.392794 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.392742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:41.393262 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.392820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:41.393262 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.392900    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:41.393262 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.392951    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:57.392934511 +0000 UTC m=+35.816118397 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:41.393262 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.392951    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:41.393262 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.393014    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:57.392997675 +0000 UTC m=+35.816181547 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:41.493933 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.493903    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:41.494083 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.494038    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:41.494083 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.494055    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:41.494083 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.494063    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:41.494243 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.494117    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:56:57.494099537 +0000 UTC m=+35.917283425 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:41.648105 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.648023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:41.648105 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.648023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:41.648312 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:41.648038    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:41.648312 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.648195    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:41.648312 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.648203    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:41.648312 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:41.648260    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:43.648045 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:43.648010    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:43.648441 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:43.648105    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:43.648441 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:43.648164    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:43.648441 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:43.648238    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:43.648441 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:43.648339    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:43.648640 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:43.648472    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:43.745836 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:43.745747    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wg4sr" event={"ID":"8dfb0243-cd9b-4843-b14a-4bde217501c8","Type":"ContainerStarted","Data":"1745ab1005d285d696d0a2ba377de9940dedd2b1e149c1a7d75429894b8c3adf"}
Sep 17 17:56:44.749713 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.749304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq" event={"ID":"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c","Type":"ContainerStarted","Data":"941ba7a4576a324641ed5b087dca02f3b210c159e397c90a570c0dd10218cfc9"}
Sep 17 17:56:44.750928 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.750901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c7zh2" event={"ID":"5c0b01b6-4b66-4b41-873e-e8be72a209e8","Type":"ContainerStarted","Data":"936672234adb55c5a774097a32524045b452f52ecb0eb1a105c618f87e020511"}
Sep 17 17:56:44.752569 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.752515    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-prs4c" event={"ID":"e9062e02-a63c-4406-9782-5c0095128a9e","Type":"ContainerStarted","Data":"7dca0009b075c065525ce0dc012b258e05e6f0c0020735dbb0e7de2283c7a75f"}
Sep 17 17:56:44.754711 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.754662    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bwbk5" event={"ID":"cd2d6bef-a10f-42e0-b02e-b73f3510a19b","Type":"ContainerStarted","Data":"285c8b8bb3804dd1e35156894c3c23e63bea9c13933f4eecee1535c2524e4222"}
Sep 17 17:56:44.759143 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759118    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"6362aaa4527ff63b8e697f4e8d86830e57db48c01fbf0bd564c2aa8db94843d5"}
Sep 17 17:56:44.759237 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759198    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"492b7769401e37ef6dd7abcbe85701254d4f83dddd5b38b00c869ce8b90f1f03"}
Sep 17 17:56:44.759282 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759251    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"9b9e40da20eb299d49d99e8d7ddae2fad978769354e7454acbb59c2eeff1529f"}
Sep 17 17:56:44.759323 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"89d8997da0b2d4d7161f73b229b3d140768fe9f205c6ef54ef706b13a2e24bae"}
Sep 17 17:56:44.759368 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759349    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"b0f40ec42f9de40a53a116ddf43f7af5172c6a9c7a12e418cfb4bed925fc47d8"}
Sep 17 17:56:44.759415 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.759401    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"4cc2b7206656879004392ea4bd489411d832923d70e5e6b16dce970919001d92"}
Sep 17 17:56:44.760490 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.760462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7s5jk" event={"ID":"a1515c65-4fb6-4f0a-9e62-649e22abdbff","Type":"ContainerStarted","Data":"6e50312054ae3c0576fd9c10a617a449bb9ea0ac916835e634f17f7aeba8abbc"}
Sep 17 17:56:44.761926 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.761903    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="68de39303cb8987b571d216d92483aed45e97295b11aeee5581f0f0119822833" exitCode=0
Sep 17 17:56:44.762003 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.761984    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"68de39303cb8987b571d216d92483aed45e97295b11aeee5581f0f0119822833"}
Sep 17 17:56:44.774478 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.774437    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c7zh2" podStartSLOduration=4.788254004 podStartE2EDuration="21.774424718s" totalImagesPullingTime="16.986170714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:44.774192202 +0000 UTC m=+23.197376097" watchObservedRunningTime="2026-09-17 17:56:44.774424718 +0000 UTC m=+23.197608624"
Sep 17 17:56:44.774682 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.774665    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-193.ec2.internal" podStartSLOduration=20.774659801 podStartE2EDuration="20.774659801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:28.733367819 +0000 UTC m=+7.156551715" watchObservedRunningTime="2026-09-17 17:56:44.774659801 +0000 UTC m=+23.197843695"
Sep 17 17:56:44.792515 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.792481    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wg4sr" podStartSLOduration=8.687781535 podStartE2EDuration="21.792469843s" totalImagesPullingTime="13.104688308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:44.79198573 +0000 UTC m=+23.215169625" watchObservedRunningTime="2026-09-17 17:56:44.792469843 +0000 UTC m=+23.215653738"
Sep 17 17:56:44.804208 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.804189    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 17:56:44.810343 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.810302    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-prs4c" podStartSLOduration=3.82473592 podStartE2EDuration="20.810289097s" totalImagesPullingTime="16.985553177s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:44.809589951 +0000 UTC m=+23.232773845" watchObservedRunningTime="2026-09-17 17:56:44.810289097 +0000 UTC m=+23.233472992"
Sep 17 17:56:44.831097 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.831057    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7s5jk" podStartSLOduration=4.836086288 podStartE2EDuration="21.831044966s" totalImagesPullingTime="16.994958678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:44.830921486 +0000 UTC m=+23.254105382" watchObservedRunningTime="2026-09-17 17:56:44.831044966 +0000 UTC m=+23.254228860"
Sep 17 17:56:44.893909 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:44.893868    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bwbk5" podStartSLOduration=3.720506668 podStartE2EDuration="20.893854995s" totalImagesPullingTime="17.173348327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:44.893742171 +0000 UTC m=+23.316926057" watchObservedRunningTime="2026-09-17 17:56:44.893854995 +0000 UTC m=+23.317038890"
Sep 17 17:56:45.630151 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.630043    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T17:56:44.804204663Z","UUID":"bf895a14-3cae-4b35-b5ba-81bd288d9613","Handler":null,"Name":"","Endpoint":""}
Sep 17 17:56:45.631915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.631892    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 17:56:45.631915 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.631929    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 17:56:45.648482 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.648459    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:45.648596 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.648460    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:45.648596 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:45.648579    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:45.648749 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:45.648726    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:45.648836 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.648465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:45.648888 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:45.648840    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:45.766565 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.766305    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qdf8r" event={"ID":"855b4e5c-fbda-4f66-accd-a5dbac52c006","Type":"ContainerStarted","Data":"15300d4a35fb9000b62c58be077267513af2540cb90aad0e5d46b7bbbcbc7b34"}
Sep 17 17:56:45.768458 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.768398    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq" event={"ID":"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c","Type":"ContainerStarted","Data":"580dcedb2cd4a9ec3a8e44c26904f596de44b9ae57bf20814fe02ee917dd95ad"}
Sep 17 17:56:45.783580 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:45.783543    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qdf8r" podStartSLOduration=5.796836739 podStartE2EDuration="22.783531896s" totalImagesPullingTime="16.986695157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:45.783164376 +0000 UTC m=+24.206348273" watchObservedRunningTime="2026-09-17 17:56:45.783531896 +0000 UTC m=+24.206715790"
Sep 17 17:56:46.773945 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:46.773907    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"ef994c8ff52b718050bba56e97049f9708cc3e2983745655330e8519897518b7"}
Sep 17 17:56:46.775895 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:46.775871    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq" event={"ID":"cc01f23a-d3ca-4f95-bbc5-e39abe35f39c","Type":"ContainerStarted","Data":"1ef3a360576068dda7d4a1dd2f92d2690ad365464ccb92297710f4afbf07766c"}
Sep 17 17:56:46.797498 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:46.797457    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xszlq" podStartSLOduration=5.064048365 podStartE2EDuration="23.797440678s" totalImagesPullingTime="18.733392313s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:46.796939578 +0000 UTC m=+25.220123475" watchObservedRunningTime="2026-09-17 17:56:46.797440678 +0000 UTC m=+25.220624573"
Sep 17 17:56:47.635638 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:47.635603    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:47.636208 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:47.636187    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:47.647301 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:47.647281    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:47.647416 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:47.647281    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:47.647416 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:47.647403    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:47.647508 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:47.647286    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:47.647508 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:47.647455    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:47.647594 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:47.647548    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:48.784499 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:48.784273    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" event={"ID":"8989de48-e3c3-40e1-b0cb-1db2ea9b570d","Type":"ContainerStarted","Data":"0ad055a0d8d41bae903b26d46e2157eed733bddfdf8acbbaeee312cd07913b97"}
Sep 17 17:56:48.785294 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:48.784586    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:48.785294 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:48.784639    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:48.800815 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:48.800772    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:48.820416 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:48.820365    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8" podStartSLOduration=8.784213869 podStartE2EDuration="25.82035147s" totalImagesPullingTime="17.036137601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:56:48.818595085 +0000 UTC m=+27.241779003" watchObservedRunningTime="2026-09-17 17:56:48.82035147 +0000 UTC m=+27.243535365"
Sep 17 17:56:49.650449 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:49.650216    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:49.650449 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:49.650362    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:49.650693 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:49.650490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:49.650693 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:49.650573    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:49.650693 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:49.650622    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:49.650693 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:49.650687    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:49.786996 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:49.786957    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:49.802593 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:49.802568    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:56:50.518455 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.518421    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6zqk5"]
Sep 17 17:56:50.518577 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.518507    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:50.518620 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:50.518574    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:50.520846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.520819    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qfd6w"]
Sep 17 17:56:50.520980 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.520914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:50.521043 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:50.521011    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:50.525013 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.524994    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8bqps"]
Sep 17 17:56:50.525110 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.525076    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:50.525177 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:50.525160    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:50.790472 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.790436    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="9b67696ea575b0c393c2bdba9e0eba89a072dd3e60513355ec059447286fefdc" exitCode=0
Sep 17 17:56:50.791097 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.790521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"9b67696ea575b0c393c2bdba9e0eba89a072dd3e60513355ec059447286fefdc"}
Sep 17 17:56:50.909377 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.909348    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:50.909525 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.909490    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 17:56:50.909964 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:50.909941    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wg4sr"
Sep 17 17:56:51.648156 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:51.648125    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:51.648302 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:51.648159    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:51.648302 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:51.648285    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:51.648439 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:51.648389    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:52.647392 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:52.647325    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:52.647796 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:52.647435    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:52.796798 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:52.796537    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="73117ce2098833d50d854c7da97b8cbbe560e0b4ff11c98e76dd065b21e76fe0" exitCode=0
Sep 17 17:56:52.796944 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:52.796624    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"73117ce2098833d50d854c7da97b8cbbe560e0b4ff11c98e76dd065b21e76fe0"}
Sep 17 17:56:53.650291 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:53.650263    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:53.650668 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:53.650430    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:53.650668 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:53.650630    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:53.650769 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:53.650753    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:53.801195 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:53.801169    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="8729f91a3ebb8d31ef7e242ebac2514d09394ee1fee5e561762ec9dc45082b82" exitCode=0
Sep 17 17:56:53.801335 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:53.801207    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"8729f91a3ebb8d31ef7e242ebac2514d09394ee1fee5e561762ec9dc45082b82"}
Sep 17 17:56:54.647690 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:54.647661    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:54.647854 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:54.647753    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:55.648030 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:55.648000    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:55.648420 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:55.648005    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:55.648420 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:55.648097    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6zqk5" podUID="bc3acc2b-d024-4416-936c-1d752ad0e43e"
Sep 17 17:56:55.648420 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:55.648172    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8bqps" podUID="97e76c9b-8f82-49a0-9714-fded16b71ee7"
Sep 17 17:56:56.647883 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.647854    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:56.648081 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:56.647978    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qfd6w" podUID="0b23d48e-e957-4278-b558-443a13430040"
Sep 17 17:56:56.712196 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.712163    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-193.ec2.internal" event="NodeReady"
Sep 17 17:56:56.712362 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.712290    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 17:56:56.762699 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.762665    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:56:56.781316 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.781290    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"]
Sep 17 17:56:56.781434 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.781401    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.784918 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.784708    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 17 17:56:56.786618 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.786591    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tjn7j\""
Sep 17 17:56:56.787485 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.787259    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 17 17:56:56.788601 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.788579    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 17 17:56:56.792044 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.792023    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 17 17:56:56.799332 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.799314    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"]
Sep 17 17:56:56.799466 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.799447    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:56.803121 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.803093    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 17:56:56.803203 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.803141    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 17 17:56:56.803203 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.803189    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 17:56:56.803393 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.803354    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 17:56:56.819367 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.819348    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"]
Sep 17 17:56:56.819477 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.819462    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:56.823024 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.823000    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7t7c2\""
Sep 17 17:56:56.823126 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.823068    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 17 17:56:56.842638 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.842616    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:56:56.842752 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.842663    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6ttvb"]
Sep 17 17:56:56.842752 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.842702    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:56.848514 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.848493    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 17:56:56.848913 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.848877    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 17:56:56.849255 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.849224    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 17:56:56.854806 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.854777    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 17:56:56.866934 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.866916    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"]
Sep 17 17:56:56.867038 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.866944    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"]
Sep 17 17:56:56.867038 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.866963    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"]
Sep 17 17:56:56.867038 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.866984    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qrq4m"]
Sep 17 17:56:56.867038 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.867021    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:56.875058 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.872553    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gqrn8\""
Sep 17 17:56:56.875058 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.872852    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 17:56:56.875058 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.873396    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 17:56:56.885030 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.885009    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6ttvb"]
Sep 17 17:56:56.885103 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.885044    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qrq4m"]
Sep 17 17:56:56.885150 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.885108    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:56.890808 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.890775    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 17:56:56.891129 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.891113    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 17:56:56.891245 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.891231    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 17:56:56.891655 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.891641    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nvjd9\""
Sep 17 17:56:56.906404 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906339    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hq5w\" (UniqueName: \"kubernetes.io/projected/78bbf176-3392-4d68-a9e9-5689dd895ab1-kube-api-access-7hq5w\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:56.906404 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/931189f9-3d36-4606-9e27-0d9db88ff1a7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:56.906533 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906431    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69rkc\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906533 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906483    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bbf176-3392-4d68-a9e9-5689dd895ab1-tmp\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:56.906533 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906507    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78bbf176-3392-4d68-a9e9-5689dd895ab1-klusterlet-config\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906589    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906656    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906680 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906671    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906899 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906709    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:56.906899 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:56.906730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltxdm\" (UniqueName: \"kubernetes.io/projected/931189f9-3d36-4606-9e27-0d9db88ff1a7-kube-api-access-ltxdm\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.007123 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69rkc\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007269 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bbf176-3392-4d68-a9e9-5689dd895ab1-tmp\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.007269 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007249    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78bbf176-3392-4d68-a9e9-5689dd895ab1-klusterlet-config\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.007370 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007270    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007370 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007370 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007370 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007344    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82f4c1a3-81b6-4005-b4f7-271643238f04-config-volume\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007431    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007455    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007494    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007519    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltxdm\" (UniqueName: \"kubernetes.io/projected/931189f9-3d36-4606-9e27-0d9db88ff1a7-kube-api-access-ltxdm\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.007552 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007546    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spzkc\" (UniqueName: \"kubernetes.io/projected/82f4c1a3-81b6-4005-b4f7-271643238f04-kube-api-access-spzkc\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007581    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007603    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007621    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007639    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt9ll\" (UniqueName: \"kubernetes.io/projected/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-kube-api-access-jt9ll\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007655    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4h7q\" (UniqueName: \"kubernetes.io/projected/470a672f-2cd1-4459-ad70-0d7b5b96624f-kube-api-access-k4h7q\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007675    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/78bbf176-3392-4d68-a9e9-5689dd895ab1-tmp\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007713    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hq5w\" (UniqueName: \"kubernetes.io/projected/78bbf176-3392-4d68-a9e9-5689dd895ab1-kube-api-access-7hq5w\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007763    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/470a672f-2cd1-4459-ad70-0d7b5b96624f-cert\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/931189f9-3d36-4606-9e27-0d9db88ff1a7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.007954 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.007852    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82f4c1a3-81b6-4005-b4f7-271643238f04-tmp-dir\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.008496 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.008151    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.009062 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.008739    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:56:57.009062 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.008761    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:56:57.009062 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.008822    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.009062 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.008851    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:57.508833906 +0000 UTC m=+35.932017788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:56:57.009326 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.009297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.012368 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.012338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.012462 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.012345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.012659 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.012640    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/78bbf176-3392-4d68-a9e9-5689dd895ab1-klusterlet-config\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.019334 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.019309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/931189f9-3d36-4606-9e27-0d9db88ff1a7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.026026 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.026001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.026267 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.026240    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltxdm\" (UniqueName: \"kubernetes.io/projected/931189f9-3d36-4606-9e27-0d9db88ff1a7-kube-api-access-ltxdm\") pod \"managed-serviceaccount-addon-agent-7fb99496fb-28f6k\" (UID: \"931189f9-3d36-4606-9e27-0d9db88ff1a7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.026380 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.026251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hq5w\" (UniqueName: \"kubernetes.io/projected/78bbf176-3392-4d68-a9e9-5689dd895ab1-kube-api-access-7hq5w\") pod \"klusterlet-addon-workmgr-67cf9494b4-9nbqq\" (UID: \"78bbf176-3392-4d68-a9e9-5689dd895ab1\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.027761 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.027735    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wg4sr_8dfb0243-cd9b-4843-b14a-4bde217501c8/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:57.027882 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.027766    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69rkc\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.109031 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt9ll\" (UniqueName: \"kubernetes.io/projected/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-kube-api-access-jt9ll\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109166 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4h7q\" (UniqueName: \"kubernetes.io/projected/470a672f-2cd1-4459-ad70-0d7b5b96624f-kube-api-access-k4h7q\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.109166 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109166 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109086    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.109166 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/470a672f-2cd1-4459-ad70-0d7b5b96624f-cert\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.109166 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109160    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:56:57.109427 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82f4c1a3-81b6-4005-b4f7-271643238f04-tmp-dir\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.109427 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109427 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109305    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82f4c1a3-81b6-4005-b4f7-271643238f04-config-volume\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.109427 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spzkc\" (UniqueName: \"kubernetes.io/projected/82f4c1a3-81b6-4005-b4f7-271643238f04-kube-api-access-spzkc\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.109618 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.109439    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:56:57.109618 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.109509    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:57.609491432 +0000 UTC m=+36.032675318 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:56:57.109727 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109683    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109727 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109713    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82f4c1a3-81b6-4005-b4f7-271643238f04-tmp-dir\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.109727 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109907 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109721    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.109907 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.109812    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.110099 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.110057    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/82f4c1a3-81b6-4005-b4f7-271643238f04-config-volume\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.112302 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.112282    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/470a672f-2cd1-4459-ad70-0d7b5b96624f-cert\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.112405 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.112311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.112405 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.112306    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-ca\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.112405 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.112361    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.113374 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.113350    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-hub\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.118980 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.118961    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spzkc\" (UniqueName: \"kubernetes.io/projected/82f4c1a3-81b6-4005-b4f7-271643238f04-kube-api-access-spzkc\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.122669 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.122648    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt9ll\" (UniqueName: \"kubernetes.io/projected/a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3-kube-api-access-jt9ll\") pod \"cluster-proxy-proxy-agent-dddbcc77c-mdkwr\" (UID: \"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.122989 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.122970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4h7q\" (UniqueName: \"kubernetes.io/projected/470a672f-2cd1-4459-ad70-0d7b5b96624f-kube-api-access-k4h7q\") pod \"ingress-canary-qrq4m\" (UID: \"470a672f-2cd1-4459-ad70-0d7b5b96624f\") " pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.137825 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.137600    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wg4sr_8dfb0243-cd9b-4843-b14a-4bde217501c8/konnectivity-agent/0.log" limit=1024
Sep 17 17:56:57.140303 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.140285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"
Sep 17 17:56:57.151623 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.151603    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:56:57.196087 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.196060    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qrq4m"
Sep 17 17:56:57.412204 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.412121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:57.412379 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.412225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:57.412379 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.412297    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:57.412379 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.412341    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:57.412379 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.412376    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret podName:0b23d48e-e957-4278-b558-443a13430040 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:29.412355085 +0000 UTC m=+67.835538974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret") pod "global-pull-secret-syncer-qfd6w" (UID: "0b23d48e-e957-4278-b558-443a13430040") : object "kube-system"/"original-pull-secret" not registered
Sep 17 17:56:57.412579 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.412398    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:29.412386814 +0000 UTC m=+67.835570695 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 17:56:57.448553 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.448475    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k"]
Sep 17 17:56:57.449374 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.449346    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"]
Sep 17 17:56:57.474360 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.474332    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"]
Sep 17 17:56:57.512697 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.512666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:57.512815 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.512773    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:57.512872 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512825    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:56:57.512872 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512847    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:56:57.512973 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512915    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:58.512895727 +0000 UTC m=+36.936079617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:56:57.512973 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512923    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 17:56:57.512973 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512942    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 17:56:57.512973 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512953    2566 projected.go:196] Error preparing data for projected volume kube-api-access-8znbj for pod openshift-network-diagnostics/network-check-target-6zqk5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:57.513156 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.512999    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj podName:bc3acc2b-d024-4416-936c-1d752ad0e43e nodeName:}" failed. No retries permitted until 2026-09-17 17:57:29.512986203 +0000 UTC m=+67.936170076 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8znbj" (UniqueName: "kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj") pod "network-check-target-6zqk5" (UID: "bc3acc2b-d024-4416-936c-1d752ad0e43e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 17:56:57.613768 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.613731    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:57.613928 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.613884    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:56:57.613985 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:57.613957    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:56:58.613935991 +0000 UTC m=+37.037119863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:56:57.647648 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.647627    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:56:57.647874 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.647844    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:56:57.651263 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.651243    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:56:57.651739 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.651315    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:56:57.651739 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.651504    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j7pjs\""
Sep 17 17:56:57.651739 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.651555    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jm6gd\""
Sep 17 17:56:57.651739 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:57.651586    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:56:58.522284 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.522250    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:56:58.522467 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:58.522384    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:56:58.522467 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:58.522402    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:56:58.522467 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:58.522462    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:00.522445984 +0000 UTC m=+38.945629872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:56:58.623637 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.623602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:56:58.623808 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:58.623770    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:56:58.623887 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:56:58.623855    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:00.623835464 +0000 UTC m=+39.047019351 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:56:58.648150 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.648121    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:56:58.651311 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.651288    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:56:58.794263 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:58.794234    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda89a60e4_e6f4_4e27_b7e1_022b9abaa8e3.slice/crio-5fcc8b39a3471fa3b5d5c9acae51f87efb726f117238907ded7c2614bb1b5e26 WatchSource:0}: Error finding container 5fcc8b39a3471fa3b5d5c9acae51f87efb726f117238907ded7c2614bb1b5e26: Status 404 returned error can't find the container with id 5fcc8b39a3471fa3b5d5c9acae51f87efb726f117238907ded7c2614bb1b5e26
Sep 17 17:56:58.796183 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:58.794667    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod931189f9_3d36_4606_9e27_0d9db88ff1a7.slice/crio-6cf224fe86ba6b03a711b055ac1cb6e84cbcfe2259f3794586b2104cdd6dce16 WatchSource:0}: Error finding container 6cf224fe86ba6b03a711b055ac1cb6e84cbcfe2259f3794586b2104cdd6dce16: Status 404 returned error can't find the container with id 6cf224fe86ba6b03a711b055ac1cb6e84cbcfe2259f3794586b2104cdd6dce16
Sep 17 17:56:58.796183 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:58.796140    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bbf176_3392_4d68_a9e9_5689dd895ab1.slice/crio-77c08df508862383e6e13379bf5a79bb9bbfb0c7a5893c6377cfd38ae4c9b0a4 WatchSource:0}: Error finding container 77c08df508862383e6e13379bf5a79bb9bbfb0c7a5893c6377cfd38ae4c9b0a4: Status 404 returned error can't find the container with id 77c08df508862383e6e13379bf5a79bb9bbfb0c7a5893c6377cfd38ae4c9b0a4
Sep 17 17:56:58.814245 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.814216    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerStarted","Data":"5fcc8b39a3471fa3b5d5c9acae51f87efb726f117238907ded7c2614bb1b5e26"}
Sep 17 17:56:58.815539 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.815510    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k" event={"ID":"931189f9-3d36-4606-9e27-0d9db88ff1a7","Type":"ContainerStarted","Data":"6cf224fe86ba6b03a711b055ac1cb6e84cbcfe2259f3794586b2104cdd6dce16"}
Sep 17 17:56:58.816694 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:58.816672    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq" event={"ID":"78bbf176-3392-4d68-a9e9-5689dd895ab1","Type":"ContainerStarted","Data":"77c08df508862383e6e13379bf5a79bb9bbfb0c7a5893c6377cfd38ae4c9b0a4"}
Sep 17 17:56:59.063388 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:59.063303    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qrq4m"]
Sep 17 17:56:59.068670 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:56:59.068643    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470a672f_2cd1_4459_ad70_0d7b5b96624f.slice/crio-2a70318b6305a84f4ae9c12dc26eb3076590ac235448a60ed591fda0ebc74086 WatchSource:0}: Error finding container 2a70318b6305a84f4ae9c12dc26eb3076590ac235448a60ed591fda0ebc74086: Status 404 returned error can't find the container with id 2a70318b6305a84f4ae9c12dc26eb3076590ac235448a60ed591fda0ebc74086
Sep 17 17:56:59.823605 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:59.823500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qrq4m" event={"ID":"470a672f-2cd1-4459-ad70-0d7b5b96624f","Type":"ContainerStarted","Data":"2a70318b6305a84f4ae9c12dc26eb3076590ac235448a60ed591fda0ebc74086"}
Sep 17 17:56:59.831517 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:59.831229    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="34e87e9858f9b547b84346f08ce1113637ce68f64defe2d020772ad4d9b4a134" exitCode=0
Sep 17 17:56:59.831517 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:56:59.831268    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"34e87e9858f9b547b84346f08ce1113637ce68f64defe2d020772ad4d9b4a134"}
Sep 17 17:57:00.541967 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:00.541931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:57:00.542246 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:00.542103    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:57:00.542246 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:00.542126    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:57:00.542246 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:00.542189    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:04.542169882 +0000 UTC m=+42.965353758 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:57:00.642836 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:00.642719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:57:00.643050 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:00.642926    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:57:00.643050 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:00.643031    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:04.642992307 +0000 UTC m=+43.066176185 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:57:00.854014 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:00.853978    2566 generic.go:356] "Generic (PLEG): container finished" podID="9cad02b4-c802-4273-b90c-55f9ca072e76" containerID="68da2ba20574566206cb7a04b4a207ec82bda9bb7a28d8f522f35bfb20ae6464" exitCode=0
Sep 17 17:57:00.854456 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:00.854060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerDied","Data":"68da2ba20574566206cb7a04b4a207ec82bda9bb7a28d8f522f35bfb20ae6464"}
Sep 17 17:57:01.867911 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:01.867567    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" event={"ID":"9cad02b4-c802-4273-b90c-55f9ca072e76","Type":"ContainerStarted","Data":"972fdfc95b4ae0cc6855f996f49bf494ce9ac119fdbc52eb2d927c8b6123bf95"}
Sep 17 17:57:01.905321 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:01.904490    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rtjw7" podStartSLOduration=8.078950769 podStartE2EDuration="37.904467699s" totalImagesPullingTime="29.82551693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:01.904025423 +0000 UTC m=+40.327209318" watchObservedRunningTime="2026-09-17 17:57:01.904467699 +0000 UTC m=+40.327651596"
Sep 17 17:57:04.581027 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:04.580995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:57:04.581449 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:04.581156    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:57:04.581449 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:04.581175    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:57:04.581449 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:04.581239    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:12.581221444 +0000 UTC m=+51.004405320 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:57:04.681767 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:04.681725    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:57:04.681961 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:04.681887    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:57:04.681961 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:04.681957    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:12.681937352 +0000 UTC m=+51.105121231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:57:07.883656 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.883575    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq" event={"ID":"78bbf176-3392-4d68-a9e9-5689dd895ab1","Type":"ContainerStarted","Data":"da52b2549ddc331c6d9088853e8c870496cabae220278cbd6771f4b8333875e4"}
Sep 17 17:57:07.884093 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.883771    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:57:07.885452 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.885404    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qrq4m" event={"ID":"470a672f-2cd1-4459-ad70-0d7b5b96624f","Type":"ContainerStarted","Data":"dcde5e71a305d081a7b95f9e245836db4661172c1233842ac4b79d0b09353255"}
Sep 17 17:57:07.885620 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.885606    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq"
Sep 17 17:57:07.886884 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.886858    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k" event={"ID":"931189f9-3d36-4606-9e27-0d9db88ff1a7","Type":"ContainerStarted","Data":"96360d65ddd89b46081fc32284f498a74f89d31217e034d352e6f6c2fea1802f"}
Sep 17 17:57:07.888891 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.888869    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerStarted","Data":"18a88ad308e5d62dd509e898872be7fc53a67b3647513d51aa0a7f14fdeaa5ae"}
Sep 17 17:57:07.888977 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.888904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerStarted","Data":"f0ad001bbd77af61fcf3a17e3f07fc6b920f2177d9f0e738eabb976d181d1c76"}
Sep 17 17:57:07.888977 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.888922    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerStarted","Data":"21f4cd62597b52efd7c80dc83f2017fdda2856edbff8b2284cf017d27a70eb06"}
Sep 17 17:57:07.904019 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.903978    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67cf9494b4-9nbqq" podStartSLOduration=27.303585964 podStartE2EDuration="35.903965867s" totalImagesPullingTime="8.600379903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:07.902533591 +0000 UTC m=+46.325717483" watchObservedRunningTime="2026-09-17 17:57:07.903965867 +0000 UTC m=+46.327149764"
Sep 17 17:57:07.920549 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.920519    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7fb99496fb-28f6k" podStartSLOduration=27.335579915 podStartE2EDuration="35.920506981s" totalImagesPullingTime="8.584927066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:07.920313953 +0000 UTC m=+46.343497837" watchObservedRunningTime="2026-09-17 17:57:07.920506981 +0000 UTC m=+46.343690872"
Sep 17 17:57:07.967573 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.967535    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" podStartSLOduration=27.381855996 podStartE2EDuration="35.967524102s" totalImagesPullingTime="8.585668106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:07.966475651 +0000 UTC m=+46.389659548" watchObservedRunningTime="2026-09-17 17:57:07.967524102 +0000 UTC m=+46.390707997"
Sep 17 17:57:07.984132 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:07.984081    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qrq4m" podStartSLOduration=3.675928855 podStartE2EDuration="11.984071734s" totalImagesPullingTime="8.308142879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:07.983475237 +0000 UTC m=+46.406659161" watchObservedRunningTime="2026-09-17 17:57:07.984071734 +0000 UTC m=+46.407255629"
Sep 17 17:57:12.645248 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:12.645213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:57:12.645706 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:12.645385    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:57:12.645706 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:12.645409    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:57:12.645706 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:12.645479    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:28.645457419 +0000 UTC m=+67.068641307 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:57:12.746577 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:12.746532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:57:12.746736 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:12.746649    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:57:12.746736 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:12.746700    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:28.746688118 +0000 UTC m=+67.169871991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:57:21.807707 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:21.807681    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dspj8"
Sep 17 17:57:28.664054 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:28.664017    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:57:28.664466 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:28.664129    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 17 17:57:28.664466 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:28.664141    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5d445cf7d4-cdrvr: secret "image-registry-tls" not found
Sep 17 17:57:28.664466 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:28.664214    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls podName:b94d76b5-4e5a-43c5-adf7-d584af4d6ce7 nodeName:}" failed. No retries permitted until 2026-09-17 17:58:00.664200057 +0000 UTC m=+99.087383929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls") pod "image-registry-5d445cf7d4-cdrvr" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7") : secret "image-registry-tls" not found
Sep 17 17:57:28.765030 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:28.764997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:57:28.765199 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:28.765177    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 17:57:28.765280 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:28.765268    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls podName:82f4c1a3-81b6-4005-b4f7-271643238f04 nodeName:}" failed. No retries permitted until 2026-09-17 17:58:00.765246668 +0000 UTC m=+99.188430546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls") pod "dns-default-6ttvb" (UID: "82f4c1a3-81b6-4005-b4f7-271643238f04") : secret "dns-default-metrics-tls" not found
Sep 17 17:57:29.470154 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.470103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:57:29.470364 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.470188    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:57:29.473225 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.473206    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 17:57:29.473283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.473260    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 17:57:29.480879 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:29.480856    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 17:57:29.480959 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:29.480926    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs podName:97e76c9b-8f82-49a0-9714-fded16b71ee7 nodeName:}" failed. No retries permitted until 2026-09-17 17:58:33.480906515 +0000 UTC m=+131.904090392 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs") pod "network-metrics-daemon-8bqps" (UID: "97e76c9b-8f82-49a0-9714-fded16b71ee7") : secret "metrics-daemon-secret" not found
Sep 17 17:57:29.483699 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.483680    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0b23d48e-e957-4278-b558-443a13430040-original-pull-secret\") pod \"global-pull-secret-syncer-qfd6w\" (UID: \"0b23d48e-e957-4278-b558-443a13430040\") " pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:57:29.558556 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.558527    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qfd6w"
Sep 17 17:57:29.570858 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.570838    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:57:29.574171 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.574148    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 17:57:29.584241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.584216    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 17:57:29.594585 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.594559    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8znbj\" (UniqueName: \"kubernetes.io/projected/bc3acc2b-d024-4416-936c-1d752ad0e43e-kube-api-access-8znbj\") pod \"network-check-target-6zqk5\" (UID: \"bc3acc2b-d024-4416-936c-1d752ad0e43e\") " pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:57:29.768628 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.768540    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jm6gd\""
Sep 17 17:57:29.775411 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.775393    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:57:29.839862 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.839838    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qfd6w"]
Sep 17 17:57:29.842844 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:57:29.842822    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b23d48e_e957_4278_b558_443a13430040.slice/crio-8855379da3af601f173832923399098478548795c1fc685c97f0e52f6df78b53 WatchSource:0}: Error finding container 8855379da3af601f173832923399098478548795c1fc685c97f0e52f6df78b53: Status 404 returned error can't find the container with id 8855379da3af601f173832923399098478548795c1fc685c97f0e52f6df78b53
Sep 17 17:57:29.952697 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:29.952669    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qfd6w" event={"ID":"0b23d48e-e957-4278-b558-443a13430040","Type":"ContainerStarted","Data":"8855379da3af601f173832923399098478548795c1fc685c97f0e52f6df78b53"}
Sep 17 17:57:30.046860 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:30.046831    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6zqk5"]
Sep 17 17:57:30.049850 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:57:30.049827    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3acc2b_d024_4416_936c_1d752ad0e43e.slice/crio-85404ba6e68df75a04120e18e8ab6fa6771437c9b94d58d8f3a2228aa2f959be WatchSource:0}: Error finding container 85404ba6e68df75a04120e18e8ab6fa6771437c9b94d58d8f3a2228aa2f959be: Status 404 returned error can't find the container with id 85404ba6e68df75a04120e18e8ab6fa6771437c9b94d58d8f3a2228aa2f959be
Sep 17 17:57:30.957434 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:30.957374    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6zqk5" event={"ID":"bc3acc2b-d024-4416-936c-1d752ad0e43e","Type":"ContainerStarted","Data":"85404ba6e68df75a04120e18e8ab6fa6771437c9b94d58d8f3a2228aa2f959be"}
Sep 17 17:57:34.972199 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:34.972113    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qfd6w" event={"ID":"0b23d48e-e957-4278-b558-443a13430040","Type":"ContainerStarted","Data":"bc85ba532649aee73353f8d30368a6dba4e6f69c4167db016abf5f7facc64644"}
Sep 17 17:57:34.973610 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:34.973584    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6zqk5" event={"ID":"bc3acc2b-d024-4416-936c-1d752ad0e43e","Type":"ContainerStarted","Data":"c7b9d8dea9f0b63c5c3517341367e4c4d002f447e56fbb5b25c8289291b29e04"}
Sep 17 17:57:34.973745 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:34.973637    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:57:34.992241 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:34.992198    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qfd6w" podStartSLOduration=66.256749306 podStartE2EDuration="1m10.992186807s" totalImagesPullingTime="4.735437501s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:34.991936815 +0000 UTC m=+73.415120709" watchObservedRunningTime="2026-09-17 17:57:34.992186807 +0000 UTC m=+73.415370701"
Sep 17 17:57:35.012737 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:35.012691    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6zqk5" podStartSLOduration=66.483311248 podStartE2EDuration="1m11.012678395s" totalImagesPullingTime="4.529367147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:35.012122282 +0000 UTC m=+73.435306178" watchObservedRunningTime="2026-09-17 17:57:35.012678395 +0000 UTC m=+73.435862290"
Sep 17 17:57:37.907471 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.907351    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqm4"]
Sep 17 17:57:37.925726 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.925701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:37.931896 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.931864    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 17:57:37.932029 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.932009    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vp6pv\""
Sep 17 17:57:37.933366 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.933194    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 17:57:37.933485 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.933442    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 17:57:37.934013 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.933994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 17:57:37.934650 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:37.934634    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqm4"]
Sep 17 17:57:38.030932 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.030904    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-data-volume\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.030932 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.030936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-crio-socket\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.031136 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.030967    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.031136 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.030986    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.031136 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.031054    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kg2c\" (UniqueName: \"kubernetes.io/projected/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-api-access-4kg2c\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.131892 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.131855    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kg2c\" (UniqueName: \"kubernetes.io/projected/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-api-access-4kg2c\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132061 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.131899    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-data-volume\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132061 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.131921    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-crio-socket\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132061 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.131949    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132061 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.131995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132061 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.132027    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:38.132280 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.132071    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:38.632059166 +0000 UTC m=+77.055243040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:38.132280 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.132097    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:57:38.132280 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.132128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-crio-socket\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.132280 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.132186    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:38.632160565 +0000 UTC m=+77.055344461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : configmap "kube-rbac-proxy" not found
Sep 17 17:57:38.132280 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.132265    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-data-volume\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.149536 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.149507    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kg2c\" (UniqueName: \"kubernetes.io/projected/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-api-access-4kg2c\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.635238 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.635203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.635238 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:38.635241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:38.635439 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.635345    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:57:38.635439 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.635354    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:38.635439 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.635408    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:39.63539213 +0000 UTC m=+78.058576008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:38.635439 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:38.635424    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:39.63541581 +0000 UTC m=+78.058599683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : configmap "kube-rbac-proxy" not found
Sep 17 17:57:39.641562 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:39.641525    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:39.641562 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:39.641564    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:39.642019 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:39.641637    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:57:39.642019 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:39.641683    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:39.642019 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:39.641758    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:41.641740352 +0000 UTC m=+80.064924248 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:39.642019 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:39.641775    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:41.641767773 +0000 UTC m=+80.064951646 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : configmap "kube-rbac-proxy" not found
Sep 17 17:57:40.619953 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:40.619922    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7s5jk_a1515c65-4fb6-4f0a-9e62-649e22abdbff/dns-node-resolver/0.log"
Sep 17 17:57:41.656881 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:41.656851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:41.656881 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:41.656884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:41.657267 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:41.656968    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:57:41.657267 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:41.656990    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:41.657267 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:41.657018    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:45.657005055 +0000 UTC m=+84.080188928 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : configmap "kube-rbac-proxy" not found
Sep 17 17:57:41.657267 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:41.657035    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:45.657022841 +0000 UTC m=+84.080206717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:41.822567 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:41.822541    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-prs4c_e9062e02-a63c-4406-9782-5c0095128a9e/node-ca/0.log"
Sep 17 17:57:42.628729 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:42.628680    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qrq4m_470a672f-2cd1-4459-ad70-0d7b5b96624f/serve-healthcheck-canary/0.log"
Sep 17 17:57:45.687561 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:45.687526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:45.687561 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:45.687562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:45.688038 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:45.687634    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 17:57:45.688038 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:45.687684    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:45.688038 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:45.687702    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:53.687688888 +0000 UTC m=+92.110872762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : configmap "kube-rbac-proxy" not found
Sep 17 17:57:45.688038 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:57:45.687747    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls podName:6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82 nodeName:}" failed. No retries permitted until 2026-09-17 17:57:53.687730142 +0000 UTC m=+92.110914037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8mqm4" (UID: "6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82") : secret "insights-runtime-extractor-tls" not found
Sep 17 17:57:53.756749 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:53.756697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:53.756749 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:53.756751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:53.757359 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:53.757338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:53.759115 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:53.759096    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8mqm4\" (UID: \"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82\") " pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:53.835069 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:53.835038    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8mqm4"
Sep 17 17:57:54.132755 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:54.132730    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8mqm4"]
Sep 17 17:57:54.135242 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:57:54.135214    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a7bc5eb_2132_4c1e_80c8_5ab8f5d78d82.slice/crio-789778942c7e1dad4a1b50688da7dbaad6d310be8f6a1c59c6cf4a4aeea06f01 WatchSource:0}: Error finding container 789778942c7e1dad4a1b50688da7dbaad6d310be8f6a1c59c6cf4a4aeea06f01: Status 404 returned error can't find the container with id 789778942c7e1dad4a1b50688da7dbaad6d310be8f6a1c59c6cf4a4aeea06f01
Sep 17 17:57:55.032327 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:55.032231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqm4" event={"ID":"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82","Type":"ContainerStarted","Data":"8e48353cfd80abd2b1b8bb28f15c847957c69044b45a7635b0cf47a0c1f009f6"}
Sep 17 17:57:55.032327 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:55.032292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqm4" event={"ID":"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82","Type":"ContainerStarted","Data":"29f58fe1b9fd3d5219b62b5650fda5eae6c92f6b96e7255573db8bebd0b6c894"}
Sep 17 17:57:55.032327 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:55.032322    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqm4" event={"ID":"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82","Type":"ContainerStarted","Data":"789778942c7e1dad4a1b50688da7dbaad6d310be8f6a1c59c6cf4a4aeea06f01"}
Sep 17 17:57:57.041633 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:57.041603    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8mqm4" event={"ID":"6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82","Type":"ContainerStarted","Data":"9bc64c94d00cc090b889f6bb87334ef2f05b7e366fb9bcf161d37db9afc6ec4e"}
Sep 17 17:57:57.073173 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:57:57.073129    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8mqm4" podStartSLOduration=18.294468189 podStartE2EDuration="20.073114879s" totalImagesPullingTime="1.77864669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 17:57:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:57:57.07264006 +0000 UTC m=+95.495823955" watchObservedRunningTime="2026-09-17 17:57:57.073114879 +0000 UTC m=+95.496298770"
Sep 17 17:58:00.709365 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:00.709322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:00.711870 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:00.711836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"image-registry-5d445cf7d4-cdrvr\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") " pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:00.810177 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:00.810140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:58:00.812522 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:00.812499    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/82f4c1a3-81b6-4005-b4f7-271643238f04-metrics-tls\") pod \"dns-default-6ttvb\" (UID: \"82f4c1a3-81b6-4005-b4f7-271643238f04\") " pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:58:00.998158 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:00.998077    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tjn7j\""
Sep 17 17:58:01.005828 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:01.005811    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:01.094103 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:01.094077    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gqrn8\""
Sep 17 17:58:01.101593 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:01.101572    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:58:01.292414 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:01.292389    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:58:01.295082 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:58:01.295059    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94d76b5_4e5a_43c5_adf7_d584af4d6ce7.slice/crio-6f4c09b1d384634f12ee81fe66f414813464069b081f8761dca7029a8a36a72d WatchSource:0}: Error finding container 6f4c09b1d384634f12ee81fe66f414813464069b081f8761dca7029a8a36a72d: Status 404 returned error can't find the container with id 6f4c09b1d384634f12ee81fe66f414813464069b081f8761dca7029a8a36a72d
Sep 17 17:58:01.376371 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:01.376352    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6ttvb"]
Sep 17 17:58:01.378363 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:58:01.378340    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f4c1a3_81b6_4005_b4f7_271643238f04.slice/crio-126d496356e9c3479b344e4631894ca3a46127fda65d619c597f7de1374a1cd7 WatchSource:0}: Error finding container 126d496356e9c3479b344e4631894ca3a46127fda65d619c597f7de1374a1cd7: Status 404 returned error can't find the container with id 126d496356e9c3479b344e4631894ca3a46127fda65d619c597f7de1374a1cd7
Sep 17 17:58:02.057764 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:02.057721    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ttvb" event={"ID":"82f4c1a3-81b6-4005-b4f7-271643238f04","Type":"ContainerStarted","Data":"126d496356e9c3479b344e4631894ca3a46127fda65d619c597f7de1374a1cd7"}
Sep 17 17:58:02.059204 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:02.059181    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" event={"ID":"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7","Type":"ContainerStarted","Data":"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"}
Sep 17 17:58:02.059319 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:02.059223    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" event={"ID":"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7","Type":"ContainerStarted","Data":"6f4c09b1d384634f12ee81fe66f414813464069b081f8761dca7029a8a36a72d"}
Sep 17 17:58:02.059319 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:02.059259    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:02.095688 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:02.095639    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" podStartSLOduration=98.095622133 podStartE2EDuration="1m38.095622133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:58:02.094389948 +0000 UTC m=+100.517573842" watchObservedRunningTime="2026-09-17 17:58:02.095622133 +0000 UTC m=+100.518806028"
Sep 17 17:58:04.068090 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:04.068060    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ttvb" event={"ID":"82f4c1a3-81b6-4005-b4f7-271643238f04","Type":"ContainerStarted","Data":"f04643a97f7ac1688476fdd4070358f61e07c5cfe11b814f92464a859a3c2191"}
Sep 17 17:58:04.068442 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:04.068137    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6ttvb" event={"ID":"82f4c1a3-81b6-4005-b4f7-271643238f04","Type":"ContainerStarted","Data":"138b86bdab230e77363fe501d830525923bb2b48cbffd9a810e8a3e6c9211145"}
Sep 17 17:58:04.068442 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:04.068201    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:58:04.094909 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:04.094857    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6ttvb" podStartSLOduration=66.213114305 podStartE2EDuration="1m8.094840552s" totalImagesPullingTime="1.881726247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:58:04.09348082 +0000 UTC m=+102.516664715" watchObservedRunningTime="2026-09-17 17:58:04.094840552 +0000 UTC m=+102.518024448"
Sep 17 17:58:05.978818 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:05.978777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6zqk5"
Sep 17 17:58:14.074070 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:14.074042    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6ttvb"
Sep 17 17:58:15.812407 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.812375    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-96427"]
Sep 17 17:58:15.815613 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.815593    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.820867 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.820846    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pk69j\""
Sep 17 17:58:15.820987 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.820934    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 17:58:15.821030 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.820986    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 17:58:15.821285 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.821269    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 17:58:15.821326 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.821269    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 17:58:15.821326 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.821275    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 17:58:15.826446 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.826430    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 17:58:15.918347 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918311    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-tls\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918347 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918352    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-sys\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918576 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-textfile\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918576 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-root\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918576 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918481    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918576 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918535    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj7tg\" (UniqueName: \"kubernetes.io/projected/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-kube-api-access-wj7tg\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918769 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918592    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-wtmp\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918769 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918657    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-metrics-client-ca\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:15.918769 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:15.918689    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-accelerators-collector-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019727 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-tls\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019739    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-sys\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-textfile\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-root\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019830    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019846    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj7tg\" (UniqueName: \"kubernetes.io/projected/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-kube-api-access-wj7tg\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019865    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-wtmp\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.019905 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-metrics-client-ca\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020240 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019929    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-accelerators-collector-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020240 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.019947    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-root\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020240 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.020046    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-wtmp\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020240 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.020095    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-sys\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020240 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.020121    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-textfile\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020437 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.020415    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-accelerators-collector-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.020564 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.020546    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-metrics-client-ca\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.022094 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.022075    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.022192 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.022175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-node-exporter-tls\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.030161 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.030139    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj7tg\" (UniqueName: \"kubernetes.io/projected/057bcc10-81c2-4d75-90a8-3ea8a5c59c30-kube-api-access-wj7tg\") pod \"node-exporter-96427\" (UID: \"057bcc10-81c2-4d75-90a8-3ea8a5c59c30\") " pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.124283 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:16.124216    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-96427"
Sep 17 17:58:16.134545 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:58:16.134519    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod057bcc10_81c2_4d75_90a8_3ea8a5c59c30.slice/crio-fd506a9bdb65fea149145045a4dd99c2e43cc1fc00d145ba59e9dc5ae278e301 WatchSource:0}: Error finding container fd506a9bdb65fea149145045a4dd99c2e43cc1fc00d145ba59e9dc5ae278e301: Status 404 returned error can't find the container with id fd506a9bdb65fea149145045a4dd99c2e43cc1fc00d145ba59e9dc5ae278e301
Sep 17 17:58:17.109617 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:17.109590    2566 generic.go:356] "Generic (PLEG): container finished" podID="057bcc10-81c2-4d75-90a8-3ea8a5c59c30" containerID="6eb8919859f4f406979e5e97d38f5a2cb148b5411e7f8d1a5508edaaba636b17" exitCode=0
Sep 17 17:58:17.109945 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:17.109649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96427" event={"ID":"057bcc10-81c2-4d75-90a8-3ea8a5c59c30","Type":"ContainerDied","Data":"6eb8919859f4f406979e5e97d38f5a2cb148b5411e7f8d1a5508edaaba636b17"}
Sep 17 17:58:17.109945 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:17.109700    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96427" event={"ID":"057bcc10-81c2-4d75-90a8-3ea8a5c59c30","Type":"ContainerStarted","Data":"fd506a9bdb65fea149145045a4dd99c2e43cc1fc00d145ba59e9dc5ae278e301"}
Sep 17 17:58:17.153302 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:17.153270    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" podUID="a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 17:58:18.117438 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:18.117405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96427" event={"ID":"057bcc10-81c2-4d75-90a8-3ea8a5c59c30","Type":"ContainerStarted","Data":"fb99d62e035f5d4d1ef845d214d67afc5e849c45919b1188050c945ebb94b43c"}
Sep 17 17:58:18.117847 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:18.117460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-96427" event={"ID":"057bcc10-81c2-4d75-90a8-3ea8a5c59c30","Type":"ContainerStarted","Data":"932d8eb9e89f368e69cbad5feb93739c32625cc368850d69db0aee06ca113c59"}
Sep 17 17:58:18.155223 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:18.155175    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-96427" podStartSLOduration=2.298866594 podStartE2EDuration="3.155159387s" totalImagesPullingTime="794.124358ms" totalInitContainerRuntime="62.168435ms" isStatefulPod=true podCreationTimestamp="2026-09-17 17:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:58:18.15486601 +0000 UTC m=+116.578049898" watchObservedRunningTime="2026-09-17 17:58:18.155159387 +0000 UTC m=+116.578343283"
Sep 17 17:58:21.010823 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:21.010776    2566 patch_prober.go:28] interesting pod/image-registry-5d445cf7d4-cdrvr container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 17 17:58:21.011242 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:21.010839    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 17:58:23.067722 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:23.067691    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:24.769872 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:24.769838    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:58:27.153054 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:27.153017    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" podUID="a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 17:58:33.549534 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:33.549493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:58:33.551845 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:33.551815    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97e76c9b-8f82-49a0-9714-fded16b71ee7-metrics-certs\") pod \"network-metrics-daemon-8bqps\" (UID: \"97e76c9b-8f82-49a0-9714-fded16b71ee7\") " pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:58:33.661832 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:33.661800    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j7pjs\""
Sep 17 17:58:33.669631 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:33.669613    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8bqps"
Sep 17 17:58:33.946718 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:33.946695    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8bqps"]
Sep 17 17:58:33.948864 ip-10-0-135-193 kubenswrapper[2566]: W0917 17:58:33.948833    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97e76c9b_8f82_49a0_9714_fded16b71ee7.slice/crio-74af305acb0c617b2fc6b80db537f9c87d34e312da28a422f7d0bf71491ed022 WatchSource:0}: Error finding container 74af305acb0c617b2fc6b80db537f9c87d34e312da28a422f7d0bf71491ed022: Status 404 returned error can't find the container with id 74af305acb0c617b2fc6b80db537f9c87d34e312da28a422f7d0bf71491ed022
Sep 17 17:58:34.167850 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:34.167816    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bqps" event={"ID":"97e76c9b-8f82-49a0-9714-fded16b71ee7","Type":"ContainerStarted","Data":"74af305acb0c617b2fc6b80db537f9c87d34e312da28a422f7d0bf71491ed022"}
Sep 17 17:58:35.173252 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:35.173157    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bqps" event={"ID":"97e76c9b-8f82-49a0-9714-fded16b71ee7","Type":"ContainerStarted","Data":"1b7de26aceea338c0eba702df583215039c4e061c13bf1142fd9a2fefe3b1bfc"}
Sep 17 17:58:35.173252 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:35.173247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8bqps" event={"ID":"97e76c9b-8f82-49a0-9714-fded16b71ee7","Type":"ContainerStarted","Data":"4ff4277cc4c1cb7b335a1ae9e95fa582efeb4a52a62122f8f235a3f9b0a0968d"}
Sep 17 17:58:35.195596 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:35.195552    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8bqps" podStartSLOduration=130.277516835 podStartE2EDuration="2m11.195538279s" totalImagesPullingTime="918.021444ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 17:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 17:58:35.195445634 +0000 UTC m=+133.618629530" watchObservedRunningTime="2026-09-17 17:58:35.195538279 +0000 UTC m=+133.618722173"
Sep 17 17:58:37.153305 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:37.153258    2566 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" podUID="a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 17 17:58:37.153859 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:37.153340    2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr"
Sep 17 17:58:37.153992 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:37.153960    2566 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"18a88ad308e5d62dd509e898872be7fc53a67b3647513d51aa0a7f14fdeaa5ae"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 17 17:58:37.154052 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:37.154023    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" podUID="a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3" containerName="service-proxy" containerID="cri-o://18a88ad308e5d62dd509e898872be7fc53a67b3647513d51aa0a7f14fdeaa5ae" gracePeriod=30
Sep 17 17:58:38.185568 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:38.185532    2566 generic.go:356] "Generic (PLEG): container finished" podID="a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3" containerID="18a88ad308e5d62dd509e898872be7fc53a67b3647513d51aa0a7f14fdeaa5ae" exitCode=2
Sep 17 17:58:38.185568 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:38.185563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerDied","Data":"18a88ad308e5d62dd509e898872be7fc53a67b3647513d51aa0a7f14fdeaa5ae"}
Sep 17 17:58:38.186075 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:38.185608    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dddbcc77c-mdkwr" event={"ID":"a89a60e4-e6f4-4e27-b7e1-022b9abaa8e3","Type":"ContainerStarted","Data":"3edd1931401c8e9fb7eec89c50980f20adf4a786335783090217690cf02cef88"}
Sep 17 17:58:49.788328 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:49.788290    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerName="registry" containerID="cri-o://90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d" gracePeriod=30
Sep 17 17:58:50.035903 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.035880    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:50.068653 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068595    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068653 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068641    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69rkc\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068670    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068694    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068721    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068748    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068803    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.068846 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.068840    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted\") pod \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\" (UID: \"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7\") "
Sep 17 17:58:50.069150 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.069060    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:58:50.069624 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.069590    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 17:58:50.071296 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.071269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc" (OuterVolumeSpecName: "kube-api-access-69rkc") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "kube-api-access-69rkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:58:50.071407 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.071312    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:58:50.071624 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.071599    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 17:58:50.071736 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.071626    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:58:50.071736 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.071624    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 17:58:50.078293 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.078267    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" (UID: "b94d76b5-4e5a-43c5-adf7-d584af4d6ce7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 17 17:58:50.169933 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169900    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-installation-pull-secrets\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.169933 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169928    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-ca-trust-extracted\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.169933 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169938    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-trusted-ca\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.170155 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169946    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69rkc\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-kube-api-access-69rkc\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.170155 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169955    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-image-registry-private-configuration\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.170155 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169964    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.170155 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169973    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-bound-sa-token\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.170155 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.169983    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7-registry-certificates\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 17:58:50.225975 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.225941    2566 generic.go:356] "Generic (PLEG): container finished" podID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerID="90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d" exitCode=0
Sep 17 17:58:50.226111 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.226005    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr"
Sep 17 17:58:50.226111 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.226029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" event={"ID":"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7","Type":"ContainerDied","Data":"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"}
Sep 17 17:58:50.226111 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.226084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5d445cf7d4-cdrvr" event={"ID":"b94d76b5-4e5a-43c5-adf7-d584af4d6ce7","Type":"ContainerDied","Data":"6f4c09b1d384634f12ee81fe66f414813464069b081f8761dca7029a8a36a72d"}
Sep 17 17:58:50.226214 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.226117    2566 scope.go:117] "RemoveContainer" containerID="90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"
Sep 17 17:58:50.234853 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.234836    2566 scope.go:117] "RemoveContainer" containerID="90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"
Sep 17 17:58:50.235126 ip-10-0-135-193 kubenswrapper[2566]: E0917 17:58:50.235105    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d\": container with ID starting with 90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d not found: ID does not exist" containerID="90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"
Sep 17 17:58:50.235180 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.235133    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d"} err="failed to get container status \"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d\": rpc error: code = NotFound desc = could not find container \"90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d\": container with ID starting with 90b15b49dc6b9dc36dd50df3e46261e8acbc7becd247601cd4efb665dbed0f3d not found: ID does not exist"
Sep 17 17:58:50.281055 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.281027    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:58:50.294611 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:50.294591    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5d445cf7d4-cdrvr"]
Sep 17 17:58:51.652314 ip-10-0-135-193 kubenswrapper[2566]: I0917 17:58:51.652282    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" path="/var/lib/kubelet/pods/b94d76b5-4e5a-43c5-adf7-d584af4d6ce7/volumes"
Sep 17 18:01:23.567758 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:23.567718    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 18:01:45.725849 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.725816    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-25ql8"]
Sep 17 18:01:45.728201 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.726026    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerName="registry"
Sep 17 18:01:45.728201 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.726036    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerName="registry"
Sep 17 18:01:45.728201 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.726074    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94d76b5-4e5a-43c5-adf7-d584af4d6ce7" containerName="registry"
Sep 17 18:01:45.729090 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.729076    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:45.732300 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.732275    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 18:01:45.732691 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.732674    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 18:01:45.733588 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.733553    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 18:01:45.733679 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.733559    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-8z42h\""
Sep 17 18:01:45.733679 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.733597    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 18:01:45.738879 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.738853    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-25ql8"]
Sep 17 18:01:45.819047 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.819020    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngx4m\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-kube-api-access-ngx4m\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:45.819180 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.819055    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:45.919333 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.919304    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngx4m\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-kube-api-access-ngx4m\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:45.919333 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.919337    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:45.919497 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:01:45.919440    2566 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Sep 17 18:01:45.919497 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:01:45.919455    2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-6c4d864b86-25ql8: secret "keda-admission-webhooks-certs" not found
Sep 17 18:01:45.919560 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:01:45.919512    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates podName:7d20c886-5252-441f-85f4-60e741e56bbb nodeName:}" failed. No retries permitted until 2026-09-17 18:01:46.41949664 +0000 UTC m=+324.842680513 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates") pod "keda-admission-6c4d864b86-25ql8" (UID: "7d20c886-5252-441f-85f4-60e741e56bbb") : secret "keda-admission-webhooks-certs" not found
Sep 17 18:01:45.930141 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:45.930117    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngx4m\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-kube-api-access-ngx4m\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:46.422494 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:46.422461    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:46.424908 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:46.424891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7d20c886-5252-441f-85f4-60e741e56bbb-certificates\") pod \"keda-admission-6c4d864b86-25ql8\" (UID: \"7d20c886-5252-441f-85f4-60e741e56bbb\") " pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:46.643653 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:46.643596    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:46.916248 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:46.916217    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-25ql8"]
Sep 17 18:01:46.919391 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:01:46.919361    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d20c886_5252_441f_85f4_60e741e56bbb.slice/crio-6c152dc1dd69d09c425130572e29ee8c4e99ad0aa279ad7ae37ad511e8314ab4 WatchSource:0}: Error finding container 6c152dc1dd69d09c425130572e29ee8c4e99ad0aa279ad7ae37ad511e8314ab4: Status 404 returned error can't find the container with id 6c152dc1dd69d09c425130572e29ee8c4e99ad0aa279ad7ae37ad511e8314ab4
Sep 17 18:01:46.920683 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:46.920667    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 18:01:47.782325 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:47.782292    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-25ql8" event={"ID":"7d20c886-5252-441f-85f4-60e741e56bbb","Type":"ContainerStarted","Data":"6c152dc1dd69d09c425130572e29ee8c4e99ad0aa279ad7ae37ad511e8314ab4"}
Sep 17 18:01:50.793502 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:50.793469    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-25ql8" event={"ID":"7d20c886-5252-441f-85f4-60e741e56bbb","Type":"ContainerStarted","Data":"4f329bec09e475a1ab4c4f6bf43851650edd14fc4670eafb08bb46ebd15e2092"}
Sep 17 18:01:50.793898 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:50.793598    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:01:50.822391 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:01:50.822338    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-25ql8" podStartSLOduration=2.910589045 podStartE2EDuration="5.822323659s" totalImagesPullingTime="2.911734614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:01:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:01:50.822193453 +0000 UTC m=+329.245377351" watchObservedRunningTime="2026-09-17 18:01:50.822323659 +0000 UTC m=+329.245507553"
Sep 17 18:02:11.798967 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:02:11.798889    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-25ql8"
Sep 17 18:03:08.919598 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.919568    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-cxw8b"]
Sep 17 18:03:08.922532 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.922514    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:08.925868 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.925842    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Sep 17 18:03:08.926554 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.926540    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 18:03:08.927132 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.927110    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-6krwt\""
Sep 17 18:03:08.927377 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.927349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 18:03:08.937165 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.937146    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-cxw8b"]
Sep 17 18:03:08.959500 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.959476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ef490b6b-72b2-4827-801d-da8172b303cf-data\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:08.959590 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:08.959502    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6vcr\" (UniqueName: \"kubernetes.io/projected/ef490b6b-72b2-4827-801d-da8172b303cf-kube-api-access-j6vcr\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.060084 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.060060    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ef490b6b-72b2-4827-801d-da8172b303cf-data\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.060192 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.060090    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6vcr\" (UniqueName: \"kubernetes.io/projected/ef490b6b-72b2-4827-801d-da8172b303cf-kube-api-access-j6vcr\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.060445 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.060425    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ef490b6b-72b2-4827-801d-da8172b303cf-data\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.077191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.077168    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6vcr\" (UniqueName: \"kubernetes.io/projected/ef490b6b-72b2-4827-801d-da8172b303cf-kube-api-access-j6vcr\") pod \"seaweedfs-64568bcd49-cxw8b\" (UID: \"ef490b6b-72b2-4827-801d-da8172b303cf\") " pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.230818 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.230721    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:09.515099 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:09.515029    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-cxw8b"]
Sep 17 18:03:09.518237 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:03:09.518208    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef490b6b_72b2_4827_801d_da8172b303cf.slice/crio-9afea634a73ed6592f8867a63f80b61c2f993f55b01c878afbefaf083a0b8842 WatchSource:0}: Error finding container 9afea634a73ed6592f8867a63f80b61c2f993f55b01c878afbefaf083a0b8842: Status 404 returned error can't find the container with id 9afea634a73ed6592f8867a63f80b61c2f993f55b01c878afbefaf083a0b8842
Sep 17 18:03:10.063419 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:10.063383    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-cxw8b" event={"ID":"ef490b6b-72b2-4827-801d-da8172b303cf","Type":"ContainerStarted","Data":"9afea634a73ed6592f8867a63f80b61c2f993f55b01c878afbefaf083a0b8842"}
Sep 17 18:03:13.075620 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:13.075581    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-cxw8b" event={"ID":"ef490b6b-72b2-4827-801d-da8172b303cf","Type":"ContainerStarted","Data":"db1f951b56c9872468927aaee917d5a0b39f0a5b936aba8febe98c55a90249d7"}
Sep 17 18:03:13.075971 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:13.075714    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:19.081395 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:19.081364    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-cxw8b"
Sep 17 18:03:19.101889 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:03:19.101847    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-cxw8b" podStartSLOduration=8.291280539 podStartE2EDuration="11.101834253s" totalImagesPullingTime="2.810553714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:03:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:03:13.095146965 +0000 UTC m=+411.518330861" watchObservedRunningTime="2026-09-17 18:03:19.101834253 +0000 UTC m=+417.525018148"
Sep 17 18:04:27.449008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.448977    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-9hl52"]
Sep 17 18:04:27.452052 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.452031    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.456665 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.456648    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Sep 17 18:04:27.456754 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.456697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-pms9x\""
Sep 17 18:04:27.467522 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.467502    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-9hl52"]
Sep 17 18:04:27.591976 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.591945    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4w7c\" (UniqueName: \"kubernetes.io/projected/79d7922c-3d2d-420e-b331-79336c240a64-kube-api-access-l4w7c\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.592137 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.592010    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79d7922c-3d2d-420e-b331-79336c240a64-tls-certs\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.693039 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.693012    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79d7922c-3d2d-420e-b331-79336c240a64-tls-certs\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.693215 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.693059    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4w7c\" (UniqueName: \"kubernetes.io/projected/79d7922c-3d2d-420e-b331-79336c240a64-kube-api-access-l4w7c\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.695516 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.695491    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79d7922c-3d2d-420e-b331-79336c240a64-tls-certs\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.706048 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.705990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4w7c\" (UniqueName: \"kubernetes.io/projected/79d7922c-3d2d-420e-b331-79336c240a64-kube-api-access-l4w7c\") pod \"model-serving-api-7b954ffc95-9hl52\" (UID: \"79d7922c-3d2d-420e-b331-79336c240a64\") " pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:27.761376 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:27.761354    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:28.029801 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:28.029763    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-9hl52"]
Sep 17 18:04:28.032351 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:04:28.032324    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d7922c_3d2d_420e_b331_79336c240a64.slice/crio-f27c47d4f32646ab7db0ef6b3cce3747dce6e87fb4e3a0ce7639f944122c29d9 WatchSource:0}: Error finding container f27c47d4f32646ab7db0ef6b3cce3747dce6e87fb4e3a0ce7639f944122c29d9: Status 404 returned error can't find the container with id f27c47d4f32646ab7db0ef6b3cce3747dce6e87fb4e3a0ce7639f944122c29d9
Sep 17 18:04:28.320442 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:28.320407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-9hl52" event={"ID":"79d7922c-3d2d-420e-b331-79336c240a64","Type":"ContainerStarted","Data":"f27c47d4f32646ab7db0ef6b3cce3747dce6e87fb4e3a0ce7639f944122c29d9"}
Sep 17 18:04:30.333147 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:30.333111    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-9hl52" event={"ID":"79d7922c-3d2d-420e-b331-79336c240a64","Type":"ContainerStarted","Data":"05c636c5432e0b21d291269f4834afa80260cf24eacba0865a7d5fac808bbbd3"}
Sep 17 18:04:30.333543 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:30.333212    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:04:30.355264 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:30.355211    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-9hl52" podStartSLOduration=1.215640463 podStartE2EDuration="3.35519271s" totalImagesPullingTime="2.139552247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:04:30.355075857 +0000 UTC m=+488.778259752" watchObservedRunningTime="2026-09-17 18:04:30.35519271 +0000 UTC m=+488.778376605"
Sep 17 18:04:41.340726 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:04:41.340696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-9hl52"
Sep 17 18:06:15.851796 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.851760    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:06:15.854863 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.854844    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:15.857666 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.857642    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-70b74-kube-rbac-proxy-sar-config\""
Sep 17 18:06:15.857761 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.857684    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 18:06:15.859003 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.858987    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-70b74-predictor-serving-cert\""
Sep 17 18:06:15.859088 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.859020    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 18:06:15.859088 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.859066    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-hn8w8\""
Sep 17 18:06:15.869531 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.869511    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:06:15.902829 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.902805    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdcnq\" (UniqueName: \"kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:15.902929 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.902849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:15.902929 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:15.902905    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.003848 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.003823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.003946 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.003864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.003946 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.003904    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdcnq\" (UniqueName: \"kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.004539 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.004520    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.006223 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.006205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.013533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.013509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdcnq\" (UniqueName: \"kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq\") pod \"success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") " pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.165291 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.165208    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:16.442970 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.442901    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:06:16.446074 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:06:16.446048    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode216d35c_d3d0_4127_9c72_860358583552.slice/crio-58eb27d4f7d8b3fd914b6f8db1a67ec4d0562b945b27e1ae676271a129a1e960 WatchSource:0}: Error finding container 58eb27d4f7d8b3fd914b6f8db1a67ec4d0562b945b27e1ae676271a129a1e960: Status 404 returned error can't find the container with id 58eb27d4f7d8b3fd914b6f8db1a67ec4d0562b945b27e1ae676271a129a1e960
Sep 17 18:06:16.684324 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:16.684285    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerStarted","Data":"58eb27d4f7d8b3fd914b6f8db1a67ec4d0562b945b27e1ae676271a129a1e960"}
Sep 17 18:06:28.734873 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:28.734850    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerStarted","Data":"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"}
Sep 17 18:06:31.748104 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:31.748056    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerStarted","Data":"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"}
Sep 17 18:06:31.748525 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:31.748273    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:31.748525 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:31.748328    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:31.749581 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:31.749557    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:06:31.772822 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:31.772757    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podStartSLOduration=2.13577838 podStartE2EDuration="16.772742127s" totalImagesPullingTime="14.636963747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:06:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:06:31.770712761 +0000 UTC m=+610.193896655" watchObservedRunningTime="2026-09-17 18:06:31.772742127 +0000 UTC m=+610.195926022"
Sep 17 18:06:32.751648 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:32.751605    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:06:37.755951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:37.755879    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:06:37.756375 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:37.756298    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:06:47.757205 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:47.757166    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:06:57.756385 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:06:57.756341    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:07:07.756904 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:07.756868    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 17 18:07:17.756937 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:17.756904    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:07:35.754065 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.754026    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:07:35.757153 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.757137    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:35.759245 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.759219    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:35.759371 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.759260    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:35.760146 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.760126    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-70b74-serving-cert\""
Sep 17 18:07:35.760216 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.760149    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-70b74-kube-rbac-proxy-sar-config\""
Sep 17 18:07:35.767242 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.767221    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:07:35.860573 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.860537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:35.860573 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.860578    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:35.860762 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:35.860693    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-70b74-serving-cert: secret "switch-graph-70b74-serving-cert" not found
Sep 17 18:07:35.860762 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:35.860752    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls podName:e44abc91-6421-4b47-aed3-ff9b669a59fe nodeName:}" failed. No retries permitted until 2026-09-17 18:07:36.360736371 +0000 UTC m=+674.783920244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls") pod "switch-graph-70b74-69b5767c67-nxls7" (UID: "e44abc91-6421-4b47-aed3-ff9b669a59fe") : secret "switch-graph-70b74-serving-cert" not found
Sep 17 18:07:35.861163 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:35.861144    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:36.364685 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.364656    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:36.367015 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.366993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") pod \"switch-graph-70b74-69b5767c67-nxls7\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") " pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:36.667435 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.667342    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:36.936042 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.935968    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:07:36.938913 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:07:36.938885    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44abc91_6421_4b47_aed3_ff9b669a59fe.slice/crio-d3ce921014a9fc839d74aee95d16226c64a5ddc5b314bf43e32379d7fe79957f WatchSource:0}: Error finding container d3ce921014a9fc839d74aee95d16226c64a5ddc5b314bf43e32379d7fe79957f: Status 404 returned error can't find the container with id d3ce921014a9fc839d74aee95d16226c64a5ddc5b314bf43e32379d7fe79957f
Sep 17 18:07:36.940908 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.940892    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 18:07:36.971990 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:36.971956    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" event={"ID":"e44abc91-6421-4b47-aed3-ff9b669a59fe","Type":"ContainerStarted","Data":"d3ce921014a9fc839d74aee95d16226c64a5ddc5b314bf43e32379d7fe79957f"}
Sep 17 18:07:39.984899 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:39.984864    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" event={"ID":"e44abc91-6421-4b47-aed3-ff9b669a59fe","Type":"ContainerStarted","Data":"514544ac528f7248acf46a936d9b42ac7b5beb2ca3b7a5a9116ad80ef31363e4"}
Sep 17 18:07:39.985290 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:39.985086    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:40.004914 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:40.004859    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podStartSLOduration=2.708010678 podStartE2EDuration="5.0048405s" totalImagesPullingTime="2.296829822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:40.003473401 +0000 UTC m=+678.426657308" watchObservedRunningTime="2026-09-17 18:07:40.0048405 +0000 UTC m=+678.428024396"
Sep 17 18:07:45.993729 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:45.993700    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:07:49.870705 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:49.870672    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:07:49.871186 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:49.870918    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" containerID="cri-o://514544ac528f7248acf46a936d9b42ac7b5beb2ca3b7a5a9116ad80ef31363e4" gracePeriod=30
Sep 17 18:07:50.028658 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.028626    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:07:50.028904 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.028878    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container" containerID="cri-o://89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95" gracePeriod=30
Sep 17 18:07:50.028968 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.028942    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kube-rbac-proxy" containerID="cri-o://fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c" gracePeriod=30
Sep 17 18:07:50.109542 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.109510    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:07:50.112894 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.112877    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.116268 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.116243    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0c0cc-kube-rbac-proxy-sar-config\""
Sep 17 18:07:50.117253 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.117236    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-0c0cc-predictor-serving-cert\""
Sep 17 18:07:50.124765 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.124706    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:07:50.165322 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.165295    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq52v\" (UniqueName: \"kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.165430 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.165328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.165430 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.165358    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.266380 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.266342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.266550 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.266421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq52v\" (UniqueName: \"kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.266550 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.266450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.266550 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:50.266509    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-serving-cert: secret "success-200-isvc-0c0cc-predictor-serving-cert" not found
Sep 17 18:07:50.266691 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:50.266597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls podName:2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5 nodeName:}" failed. No retries permitted until 2026-09-17 18:07:50.766575552 +0000 UTC m=+689.189759442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls") pod "success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" (UID: "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5") : secret "success-200-isvc-0c0cc-predictor-serving-cert" not found
Sep 17 18:07:50.267108 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.267089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.280222 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.280192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq52v\" (UniqueName: \"kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.770802 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.770745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.773177 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.773151    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") pod \"success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") " pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:50.992802 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:50.992747    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:07:51.021884 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:51.021794    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:51.025554 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:51.025524    2566 generic.go:356] "Generic (PLEG): container finished" podID="e216d35c-d3d0-4127-9c72-860358583552" containerID="fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c" exitCode=2
Sep 17 18:07:51.025653 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:51.025595    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerDied","Data":"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"}
Sep 17 18:07:51.312306 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:51.312278    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:07:51.315543 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:07:51.315519    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1a3893_b0dc_4c9a_8055_248c3cd2f4d5.slice/crio-3c7b2960bc39285ed79b4665cbb52b23981f3d2598c64fa5e2c0d33b2f7c02b3 WatchSource:0}: Error finding container 3c7b2960bc39285ed79b4665cbb52b23981f3d2598c64fa5e2c0d33b2f7c02b3: Status 404 returned error can't find the container with id 3c7b2960bc39285ed79b4665cbb52b23981f3d2598c64fa5e2c0d33b2f7c02b3
Sep 17 18:07:52.031419 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.031378    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerStarted","Data":"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"}
Sep 17 18:07:52.031843 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.031439    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerStarted","Data":"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"}
Sep 17 18:07:52.031843 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.031459    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerStarted","Data":"3c7b2960bc39285ed79b4665cbb52b23981f3d2598c64fa5e2c0d33b2f7c02b3"}
Sep 17 18:07:52.031843 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.031484    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:52.031843 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.031503    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:52.032919 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.032889    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:07:52.050770 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.050715    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podStartSLOduration=2.050697952 podStartE2EDuration="2.050697952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:07:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:07:52.050473171 +0000 UTC m=+690.473657102" watchObservedRunningTime="2026-09-17 18:07:52.050697952 +0000 UTC m=+690.473881847"
Sep 17 18:07:52.752908 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:52.752861    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.16:8643/healthz\": dial tcp 10.132.0.16:8643: connect: connection refused"
Sep 17 18:07:53.035458 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.035365    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:07:53.474671 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.474648    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:07:53.591727 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.591696    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls\") pod \"e216d35c-d3d0-4127-9c72-860358583552\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") "
Sep 17 18:07:53.591727 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.591737    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config\") pod \"e216d35c-d3d0-4127-9c72-860358583552\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") "
Sep 17 18:07:53.591948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.591761    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdcnq\" (UniqueName: \"kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq\") pod \"e216d35c-d3d0-4127-9c72-860358583552\" (UID: \"e216d35c-d3d0-4127-9c72-860358583552\") "
Sep 17 18:07:53.592102 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.592080    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e216d35c-d3d0-4127-9c72-860358583552" (UID: "e216d35c-d3d0-4127-9c72-860358583552"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:07:53.594014 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.593991    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e216d35c-d3d0-4127-9c72-860358583552" (UID: "e216d35c-d3d0-4127-9c72-860358583552"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:07:53.594123 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.594105    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq" (OuterVolumeSpecName: "kube-api-access-fdcnq") pod "e216d35c-d3d0-4127-9c72-860358583552" (UID: "e216d35c-d3d0-4127-9c72-860358583552"). InnerVolumeSpecName "kube-api-access-fdcnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:07:53.692565 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.692541    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e216d35c-d3d0-4127-9c72-860358583552-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:07:53.692697 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.692584    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e216d35c-d3d0-4127-9c72-860358583552-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:07:53.692697 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:53.692595    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdcnq\" (UniqueName: \"kubernetes.io/projected/e216d35c-d3d0-4127-9c72-860358583552-kube-api-access-fdcnq\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:07:54.040107 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.040078    2566 generic.go:356] "Generic (PLEG): container finished" podID="e216d35c-d3d0-4127-9c72-860358583552" containerID="89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95" exitCode=0
Sep 17 18:07:54.040459 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.040141    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"
Sep 17 18:07:54.040459 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.040154    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerDied","Data":"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"}
Sep 17 18:07:54.040459 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.040214    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7" event={"ID":"e216d35c-d3d0-4127-9c72-860358583552","Type":"ContainerDied","Data":"58eb27d4f7d8b3fd914b6f8db1a67ec4d0562b945b27e1ae676271a129a1e960"}
Sep 17 18:07:54.040459 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.040241    2566 scope.go:117] "RemoveContainer" containerID="fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"
Sep 17 18:07:54.047847 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.047830    2566 scope.go:117] "RemoveContainer" containerID="89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"
Sep 17 18:07:54.054689 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.054669    2566 scope.go:117] "RemoveContainer" containerID="fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"
Sep 17 18:07:54.054990 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:54.054970    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c\": container with ID starting with fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c not found: ID does not exist" containerID="fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"
Sep 17 18:07:54.055034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.054999    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c"} err="failed to get container status \"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c\": rpc error: code = NotFound desc = could not find container \"fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c\": container with ID starting with fb7fde90283a055f47d918d51c5c83c5a28225eb0bdef5ef3bb4838fcf95cf6c not found: ID does not exist"
Sep 17 18:07:54.055034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.055018    2566 scope.go:117] "RemoveContainer" containerID="89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"
Sep 17 18:07:54.055298 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:07:54.055279    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95\": container with ID starting with 89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95 not found: ID does not exist" containerID="89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"
Sep 17 18:07:54.055343 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.055305    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95"} err="failed to get container status \"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95\": rpc error: code = NotFound desc = could not find container \"89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95\": container with ID starting with 89676c7598289821b80cf914543ba8f6b7836b825565ac4b528d4086448f9d95 not found: ID does not exist"
Sep 17 18:07:54.082886 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.082862    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:07:54.085652 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:54.085632    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-70b74-predictor-5d6ff7f77f-2tqw7"]
Sep 17 18:07:55.653430 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:55.653393    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e216d35c-d3d0-4127-9c72-860358583552" path="/var/lib/kubelet/pods/e216d35c-d3d0-4127-9c72-860358583552/volumes"
Sep 17 18:07:55.991878 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:55.991778    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:07:58.039712 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:58.039686    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:07:58.040182 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:07:58.040155    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:08:00.992353 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:00.992310    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:00.992888 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:00.992480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:08:05.992625 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:05.992586    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:08.040266 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:08.040228    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:08:10.992323 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:10.992287    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:15.992351 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:15.992311    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:18.040191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:18.040153    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:08:20.133531 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.133435    2566 generic.go:356] "Generic (PLEG): container finished" podID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerID="514544ac528f7248acf46a936d9b42ac7b5beb2ca3b7a5a9116ad80ef31363e4" exitCode=0
Sep 17 18:08:20.133531 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.133480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" event={"ID":"e44abc91-6421-4b47-aed3-ff9b669a59fe","Type":"ContainerDied","Data":"514544ac528f7248acf46a936d9b42ac7b5beb2ca3b7a5a9116ad80ef31363e4"}
Sep 17 18:08:20.513952 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.513929    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:08:20.577624 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.577593    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle\") pod \"e44abc91-6421-4b47-aed3-ff9b669a59fe\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") "
Sep 17 18:08:20.577795 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.577648    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") pod \"e44abc91-6421-4b47-aed3-ff9b669a59fe\" (UID: \"e44abc91-6421-4b47-aed3-ff9b669a59fe\") "
Sep 17 18:08:20.578005 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.577983    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "e44abc91-6421-4b47-aed3-ff9b669a59fe" (UID: "e44abc91-6421-4b47-aed3-ff9b669a59fe"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:08:20.579689 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.579667    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e44abc91-6421-4b47-aed3-ff9b669a59fe" (UID: "e44abc91-6421-4b47-aed3-ff9b669a59fe"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:08:20.678754 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.678663    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e44abc91-6421-4b47-aed3-ff9b669a59fe-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:08:20.678754 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:20.678699    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e44abc91-6421-4b47-aed3-ff9b669a59fe-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:08:21.138490 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.138457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7" event={"ID":"e44abc91-6421-4b47-aed3-ff9b669a59fe","Type":"ContainerDied","Data":"d3ce921014a9fc839d74aee95d16226c64a5ddc5b314bf43e32379d7fe79957f"}
Sep 17 18:08:21.138892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.138517    2566 scope.go:117] "RemoveContainer" containerID="514544ac528f7248acf46a936d9b42ac7b5beb2ca3b7a5a9116ad80ef31363e4"
Sep 17 18:08:21.138892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.138466    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"
Sep 17 18:08:21.160979 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.160954    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:08:21.164639 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.164613    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-70b74-69b5767c67-nxls7"]
Sep 17 18:08:21.652986 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:21.652948    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" path="/var/lib/kubelet/pods/e44abc91-6421-4b47-aed3-ff9b669a59fe/volumes"
Sep 17 18:08:28.040695 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:28.040654    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 17 18:08:35.857207 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857172    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857429    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857440    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857451    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857457    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857469    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kube-rbac-proxy"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857475    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kube-rbac-proxy"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857512    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kube-rbac-proxy"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857519    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e216d35c-d3d0-4127-9c72-860358583552" containerName="kserve-container"
Sep 17 18:08:35.857660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.857524    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44abc91-6421-4b47-aed3-ff9b669a59fe" containerName="switch-graph-70b74"
Sep 17 18:08:35.861711 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.861695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:35.864545 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.864525    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\""
Sep 17 18:08:35.864545 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.864538    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\""
Sep 17 18:08:35.870691 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:35.870667    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:08:36.003583 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.003549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.003583 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.003591    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.104254 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.104222    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.104445 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.104291    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.104445 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:08:36.104396    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found
Sep 17 18:08:36.104564 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:08:36.104472    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls podName:f34177c9-972c-4d70-b86a-572e003c80ba nodeName:}" failed. No retries permitted until 2026-09-17 18:08:36.604450729 +0000 UTC m=+735.027634618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls") pod "model-chainer-f7558cd98-v8lf8" (UID: "f34177c9-972c-4d70-b86a-572e003c80ba") : secret "model-chainer-serving-cert" not found
Sep 17 18:08:36.104855 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.104836    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.607729 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.607680    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.610116 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.610089    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") pod \"model-chainer-f7558cd98-v8lf8\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") " pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:36.772208 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:36.772172    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:37.040951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:37.040921    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:08:37.044166 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:08:37.044140    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf34177c9_972c_4d70_b86a_572e003c80ba.slice/crio-5d2e9b57b26f35d115cef7dc5fdea9a123055937bff8fd9ee9e9a0a140cb72f3 WatchSource:0}: Error finding container 5d2e9b57b26f35d115cef7dc5fdea9a123055937bff8fd9ee9e9a0a140cb72f3: Status 404 returned error can't find the container with id 5d2e9b57b26f35d115cef7dc5fdea9a123055937bff8fd9ee9e9a0a140cb72f3
Sep 17 18:08:37.198905 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:37.198866    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" event={"ID":"f34177c9-972c-4d70-b86a-572e003c80ba","Type":"ContainerStarted","Data":"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"}
Sep 17 18:08:37.199088 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:37.198918    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" event={"ID":"f34177c9-972c-4d70-b86a-572e003c80ba","Type":"ContainerStarted","Data":"5d2e9b57b26f35d115cef7dc5fdea9a123055937bff8fd9ee9e9a0a140cb72f3"}
Sep 17 18:08:37.199088 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:37.199001    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:37.216708 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:37.216657    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podStartSLOduration=2.216639716 podStartE2EDuration="2.216639716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:08:37.215529086 +0000 UTC m=+735.638712992" watchObservedRunningTime="2026-09-17 18:08:37.216639716 +0000 UTC m=+735.639823616"
Sep 17 18:08:38.041602 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:38.041574    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:08:43.207771 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:43.207737    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:08:46.465497 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.465464    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:08:46.465908 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.465663    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" containerID="cri-o://2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432" gracePeriod=30
Sep 17 18:08:46.763987 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.763909    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:08:46.767139 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.767123    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.769765 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.769727    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4a77e-predictor-serving-cert\""
Sep 17 18:08:46.770013 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.769986    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4a77e-kube-rbac-proxy-sar-config\""
Sep 17 18:08:46.777432 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.777413    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:08:46.881576 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.881548    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5458\" (UniqueName: \"kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.881751 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.881583    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.881751 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.881619    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.982826 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.982766    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5458\" (UniqueName: \"kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.982826 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.982830    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.983037 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.982873    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.983037 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:08:46.982923    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-serving-cert: secret "success-200-isvc-4a77e-predictor-serving-cert" not found
Sep 17 18:08:46.983037 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:08:46.982993    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls podName:f6476fb1-0c87-4d63-9dea-a69e622d1d38 nodeName:}" failed. No retries permitted until 2026-09-17 18:08:47.48297385 +0000 UTC m=+745.906157723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls") pod "success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" (UID: "f6476fb1-0c87-4d63-9dea-a69e622d1d38") : secret "success-200-isvc-4a77e-predictor-serving-cert" not found
Sep 17 18:08:46.983450 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.983432    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:46.996486 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:46.996454    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5458\" (UniqueName: \"kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:47.487113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:47.487075    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:47.489511 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:47.489492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") pod \"success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") " pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:47.677306 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:47.677272    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:47.946379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:47.946355    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:08:47.948855 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:08:47.948826    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6476fb1_0c87_4d63_9dea_a69e622d1d38.slice/crio-ff46d10f8ee84ea5d28c4f8e65cc031f5cf9f5beb820890f3eef307be2805746 WatchSource:0}: Error finding container ff46d10f8ee84ea5d28c4f8e65cc031f5cf9f5beb820890f3eef307be2805746: Status 404 returned error can't find the container with id ff46d10f8ee84ea5d28c4f8e65cc031f5cf9f5beb820890f3eef307be2805746
Sep 17 18:08:48.206560 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.206472    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:48.236652 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.236615    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerStarted","Data":"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"}
Sep 17 18:08:48.236827 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.236684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerStarted","Data":"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"}
Sep 17 18:08:48.236827 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.236714    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerStarted","Data":"ff46d10f8ee84ea5d28c4f8e65cc031f5cf9f5beb820890f3eef307be2805746"}
Sep 17 18:08:48.236827 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.236744    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:48.256122 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:48.256074    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podStartSLOduration=2.256057008 podStartE2EDuration="2.256057008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:08:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:08:48.254165614 +0000 UTC m=+746.677349535" watchObservedRunningTime="2026-09-17 18:08:48.256057008 +0000 UTC m=+746.679240904"
Sep 17 18:08:49.240838 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:49.240810    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:49.242247 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:49.242217    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:08:50.128917 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.128886    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:08:50.131898 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.131884    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.134673 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.134638    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0c0cc-serving-cert\""
Sep 17 18:08:50.134673 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.134638    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0c0cc-kube-rbac-proxy-sar-config\""
Sep 17 18:08:50.144612 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.144588    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:08:50.243849 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.243809    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:08:50.310882 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.310851    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.311040 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.310914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.411902 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.411805    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.412060 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.412002    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.412611 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.412591    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.414146 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.414128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls\") pod \"switch-graph-0c0cc-5c697ddcd5-bzdsc\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") " pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.444164 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.444140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:50.714841 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:50.714755    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:08:50.716950 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:08:50.716923    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097f4786_8ba4_459e_9d77_73606b4db8a8.slice/crio-e09b6307a45958e8cc1a4bf77011aca7409ba3088768992e18da18b9f848a36c WatchSource:0}: Error finding container e09b6307a45958e8cc1a4bf77011aca7409ba3088768992e18da18b9f848a36c: Status 404 returned error can't find the container with id e09b6307a45958e8cc1a4bf77011aca7409ba3088768992e18da18b9f848a36c
Sep 17 18:08:51.248695 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:51.248659    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" event={"ID":"097f4786-8ba4-459e-9d77-73606b4db8a8","Type":"ContainerStarted","Data":"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"}
Sep 17 18:08:51.249111 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:51.248816    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:51.249111 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:51.248862    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" event={"ID":"097f4786-8ba4-459e-9d77-73606b4db8a8","Type":"ContainerStarted","Data":"e09b6307a45958e8cc1a4bf77011aca7409ba3088768992e18da18b9f848a36c"}
Sep 17 18:08:51.266754 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:51.266708    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podStartSLOduration=1.266693676 podStartE2EDuration="1.266693676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:08:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:08:51.265281094 +0000 UTC m=+749.688464989" watchObservedRunningTime="2026-09-17 18:08:51.266693676 +0000 UTC m=+749.689877599"
Sep 17 18:08:53.206398 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:53.206359    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:55.249648 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:55.249616    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:08:55.250174 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:55.250149    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:08:57.259185 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:57.259151    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:08:58.207244 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:58.207211    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:08:58.207437 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:08:58.207318    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:09:03.207103 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:03.207066    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:09:05.250405 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:05.250369    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:09:08.207320 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:08.207283    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:09:13.207181 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:13.207133    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:09:15.250342 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:15.250302    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:09:16.606811 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.606771    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:09:16.701871 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.701247    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle\") pod \"f34177c9-972c-4d70-b86a-572e003c80ba\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") "
Sep 17 18:09:16.701871 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.701316    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") pod \"f34177c9-972c-4d70-b86a-572e003c80ba\" (UID: \"f34177c9-972c-4d70-b86a-572e003c80ba\") "
Sep 17 18:09:16.706173 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.706135    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "f34177c9-972c-4d70-b86a-572e003c80ba" (UID: "f34177c9-972c-4d70-b86a-572e003c80ba"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:09:16.706173 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.706163    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f34177c9-972c-4d70-b86a-572e003c80ba" (UID: "f34177c9-972c-4d70-b86a-572e003c80ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:09:16.802669 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.802640    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f34177c9-972c-4d70-b86a-572e003c80ba-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:09:16.802669 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:16.802670    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f34177c9-972c-4d70-b86a-572e003c80ba-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:09:17.344381 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.344347    2566 generic.go:356] "Generic (PLEG): container finished" podID="f34177c9-972c-4d70-b86a-572e003c80ba" containerID="2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432" exitCode=0
Sep 17 18:09:17.344566 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.344411    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"
Sep 17 18:09:17.344566 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.344435    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" event={"ID":"f34177c9-972c-4d70-b86a-572e003c80ba","Type":"ContainerDied","Data":"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"}
Sep 17 18:09:17.344566 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.344522    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8" event={"ID":"f34177c9-972c-4d70-b86a-572e003c80ba","Type":"ContainerDied","Data":"5d2e9b57b26f35d115cef7dc5fdea9a123055937bff8fd9ee9e9a0a140cb72f3"}
Sep 17 18:09:17.344566 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.344550    2566 scope.go:117] "RemoveContainer" containerID="2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"
Sep 17 18:09:17.353332 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.353314    2566 scope.go:117] "RemoveContainer" containerID="2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"
Sep 17 18:09:17.353571 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:09:17.353551    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432\": container with ID starting with 2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432 not found: ID does not exist" containerID="2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"
Sep 17 18:09:17.353651 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.353578    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432"} err="failed to get container status \"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432\": rpc error: code = NotFound desc = could not find container \"2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432\": container with ID starting with 2def63eb05b9c5b5c50372a24c4fbec3ea006f4432ac19e48146ba274993a432 not found: ID does not exist"
Sep 17 18:09:17.365710 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.365687    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:09:17.369553 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.369532    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-f7558cd98-v8lf8"]
Sep 17 18:09:17.654040 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:17.653965    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" path="/var/lib/kubelet/pods/f34177c9-972c-4d70-b86a-572e003c80ba/volumes"
Sep 17 18:09:25.251080 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:25.251040    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 17 18:09:35.250936 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:35.250901    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:09:46.716136 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.716061    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:09:46.716544 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.716333    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer"
Sep 17 18:09:46.716544 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.716344    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer"
Sep 17 18:09:46.716544 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.716397    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f34177c9-972c-4d70-b86a-572e003c80ba" containerName="model-chainer"
Sep 17 18:09:46.719141 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.719122    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.722422 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.722398    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4a77e-serving-cert\""
Sep 17 18:09:46.722535 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.722399    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4a77e-kube-rbac-proxy-sar-config\""
Sep 17 18:09:46.728637 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.728614    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:09:46.823687 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.823654    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.823687 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.823691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.924982 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.924948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.925146 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.924989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.925702 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.925683    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:46.927467 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:46.927451    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls\") pod \"sequence-graph-4a77e-7897f464d9-9scnp\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") " pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:47.028919 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.028836    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:47.317708 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.317671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:09:47.321833 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:09:47.321805    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba66a72_ae61_430f_98a1_1b93315a22cb.slice/crio-82873101d8779f5b78f9c746c362473c1ba5832f46f48be60aee46050f609957 WatchSource:0}: Error finding container 82873101d8779f5b78f9c746c362473c1ba5832f46f48be60aee46050f609957: Status 404 returned error can't find the container with id 82873101d8779f5b78f9c746c362473c1ba5832f46f48be60aee46050f609957
Sep 17 18:09:47.456841 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.456810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" event={"ID":"8ba66a72-ae61-430f-98a1-1b93315a22cb","Type":"ContainerStarted","Data":"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"}
Sep 17 18:09:47.457008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.456859    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" event={"ID":"8ba66a72-ae61-430f-98a1-1b93315a22cb","Type":"ContainerStarted","Data":"82873101d8779f5b78f9c746c362473c1ba5832f46f48be60aee46050f609957"}
Sep 17 18:09:47.457008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.456920    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:09:47.474663 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:47.474611    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podStartSLOduration=1.47459417 podStartE2EDuration="1.47459417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:09:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:09:47.473410237 +0000 UTC m=+805.896594133" watchObservedRunningTime="2026-09-17 18:09:47.47459417 +0000 UTC m=+805.897778058"
Sep 17 18:09:53.465720 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:09:53.465693    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:17:04.653755 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.653720    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:17:04.654314 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.653979    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" containerID="cri-o://659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9" gracePeriod=30
Sep 17 18:17:04.768949 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.768916    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:17:04.769182 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.769152    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container" containerID="cri-o://1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57" gracePeriod=30
Sep 17 18:17:04.769270 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.769177    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kube-rbac-proxy" containerID="cri-o://b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55" gracePeriod=30
Sep 17 18:17:04.909421 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.909341    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:17:04.912529 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.912511    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:04.915016 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.914986    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b0976-predictor-serving-cert\""
Sep 17 18:17:04.915148 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.914992    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-b0976-kube-rbac-proxy-sar-config\""
Sep 17 18:17:04.924658 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.924630    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:17:04.988872 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.988840    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerID="b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55" exitCode=2
Sep 17 18:17:04.989044 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:04.988921    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerDied","Data":"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"}
Sep 17 18:17:05.007496 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.007466    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.007649 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.007500    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.007649 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.007606    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2lxd\" (UniqueName: \"kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.108031 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.107997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.108031 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.108042    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.108290 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.108131    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2lxd\" (UniqueName: \"kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.108825 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.108772    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.110426 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.110405    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.116298 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.116274    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2lxd\" (UniqueName: \"kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd\") pod \"success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") " pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.222367 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.222273    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.499545 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.499468    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:17:05.502857 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:17:05.502832    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode85e55d9_1383_4434_b8e6_63aedad2ab61.slice/crio-cb6f62b7f8119c05cf68d65be0062e9ec68fb5d513a4c2a2faf6fc839237e875 WatchSource:0}: Error finding container cb6f62b7f8119c05cf68d65be0062e9ec68fb5d513a4c2a2faf6fc839237e875: Status 404 returned error can't find the container with id cb6f62b7f8119c05cf68d65be0062e9ec68fb5d513a4c2a2faf6fc839237e875
Sep 17 18:17:05.994199 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.994167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerStarted","Data":"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"}
Sep 17 18:17:05.994601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.994221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerStarted","Data":"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"}
Sep 17 18:17:05.994601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.994248    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.994601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.994269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerStarted","Data":"cb6f62b7f8119c05cf68d65be0062e9ec68fb5d513a4c2a2faf6fc839237e875"}
Sep 17 18:17:05.994601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.994286    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:05.995738 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:05.995710    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:06.011462 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:06.011408    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podStartSLOduration=2.011391646 podStartE2EDuration="2.011391646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:17:06.010473865 +0000 UTC m=+1244.433657761" watchObservedRunningTime="2026-09-17 18:17:06.011391646 +0000 UTC m=+1244.434575540"
Sep 17 18:17:06.998385 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:06.998347    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:07.258700 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:07.258605    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:07.913856 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:07.913829    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:17:08.003474 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.003385    2566 generic.go:356] "Generic (PLEG): container finished" podID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerID="1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57" exitCode=0
Sep 17 18:17:08.003474 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.003457    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerDied","Data":"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"}
Sep 17 18:17:08.003979 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.003531    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s" event={"ID":"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5","Type":"ContainerDied","Data":"3c7b2960bc39285ed79b4665cbb52b23981f3d2598c64fa5e2c0d33b2f7c02b3"}
Sep 17 18:17:08.003979 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.003458    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"
Sep 17 18:17:08.003979 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.003559    2566 scope.go:117] "RemoveContainer" containerID="b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"
Sep 17 18:17:08.011409 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.011388    2566 scope.go:117] "RemoveContainer" containerID="1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"
Sep 17 18:17:08.018258 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.018242    2566 scope.go:117] "RemoveContainer" containerID="b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"
Sep 17 18:17:08.018549 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:17:08.018530    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55\": container with ID starting with b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55 not found: ID does not exist" containerID="b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"
Sep 17 18:17:08.018628 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.018561    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55"} err="failed to get container status \"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55\": rpc error: code = NotFound desc = could not find container \"b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55\": container with ID starting with b121a6117ed9c329a075412c8b5813363153ff30380872612b2302982f729b55 not found: ID does not exist"
Sep 17 18:17:08.018628 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.018588    2566 scope.go:117] "RemoveContainer" containerID="1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"
Sep 17 18:17:08.018893 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:17:08.018876    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57\": container with ID starting with 1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57 not found: ID does not exist" containerID="1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"
Sep 17 18:17:08.018939 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.018907    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57"} err="failed to get container status \"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57\": rpc error: code = NotFound desc = could not find container \"1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57\": container with ID starting with 1906c6840e6441ff535bea58299bda865f1676e6ce69442f28992bfa4511ec57 not found: ID does not exist"
Sep 17 18:17:08.031151 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.031131    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq52v\" (UniqueName: \"kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v\") pod \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") "
Sep 17 18:17:08.031237 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.031199    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config\") pod \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") "
Sep 17 18:17:08.031237 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.031218    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") pod \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\" (UID: \"2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5\") "
Sep 17 18:17:08.031574 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.031549    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" (UID: "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:17:08.033264 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.033238    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v" (OuterVolumeSpecName: "kube-api-access-qq52v") pod "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" (UID: "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5"). InnerVolumeSpecName "kube-api-access-qq52v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:17:08.033343 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.033286    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" (UID: "2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:17:08.132407 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.132368    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:17:08.132407 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.132400    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:17:08.132407 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.132411    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq52v\" (UniqueName: \"kubernetes.io/projected/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5-kube-api-access-qq52v\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:17:08.324923 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.324892    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:17:08.328711 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:08.328689    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-0c0cc-predictor-565c9b6cf7-chh6s"]
Sep 17 18:17:09.653801 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:09.653756    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" path="/var/lib/kubelet/pods/2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5/volumes"
Sep 17 18:17:12.004089 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:12.004062    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:17:12.004599 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:12.004532    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:12.258090 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:12.258009    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:17.258353 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:17.258314    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:17.258737 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:17.258443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:17:22.005008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:22.004965    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:22.258326 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:22.258242    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:27.257613 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:27.257571    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:32.004942 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:32.004901    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:32.258327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:32.258238    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:17:34.797631 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.797608    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:17:34.822533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.822506    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle\") pod \"097f4786-8ba4-459e-9d77-73606b4db8a8\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") "
Sep 17 18:17:34.822690 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.822580    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls\") pod \"097f4786-8ba4-459e-9d77-73606b4db8a8\" (UID: \"097f4786-8ba4-459e-9d77-73606b4db8a8\") "
Sep 17 18:17:34.822878 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.822856    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "097f4786-8ba4-459e-9d77-73606b4db8a8" (UID: "097f4786-8ba4-459e-9d77-73606b4db8a8"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:17:34.824699 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.824669    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "097f4786-8ba4-459e-9d77-73606b4db8a8" (UID: "097f4786-8ba4-459e-9d77-73606b4db8a8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:17:34.923950 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.923865    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/097f4786-8ba4-459e-9d77-73606b4db8a8-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:17:34.923950 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:34.923894    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/097f4786-8ba4-459e-9d77-73606b4db8a8-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:17:35.101611 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.101566    2566 generic.go:356] "Generic (PLEG): container finished" podID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerID="659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9" exitCode=0
Sep 17 18:17:35.101611 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.101608    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"
Sep 17 18:17:35.101879 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.101645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" event={"ID":"097f4786-8ba4-459e-9d77-73606b4db8a8","Type":"ContainerDied","Data":"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"}
Sep 17 18:17:35.101879 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.101692    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc" event={"ID":"097f4786-8ba4-459e-9d77-73606b4db8a8","Type":"ContainerDied","Data":"e09b6307a45958e8cc1a4bf77011aca7409ba3088768992e18da18b9f848a36c"}
Sep 17 18:17:35.101879 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.101720    2566 scope.go:117] "RemoveContainer" containerID="659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"
Sep 17 18:17:35.110394 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.110375    2566 scope.go:117] "RemoveContainer" containerID="659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"
Sep 17 18:17:35.110707 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:17:35.110683    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9\": container with ID starting with 659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9 not found: ID does not exist" containerID="659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"
Sep 17 18:17:35.110809 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.110716    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9"} err="failed to get container status \"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9\": rpc error: code = NotFound desc = could not find container \"659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9\": container with ID starting with 659b8bc4167c5ece09b9b7c07f4fd0c62f2fd216554cb34d87e72c19098702c9 not found: ID does not exist"
Sep 17 18:17:35.122490 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.122464    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:17:35.125630 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.125607    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0c0cc-5c697ddcd5-bzdsc"]
Sep 17 18:17:35.653125 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:35.653086    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" path="/var/lib/kubelet/pods/097f4786-8ba4-459e-9d77-73606b4db8a8/volumes"
Sep 17 18:17:42.005204 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:42.005154    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 17 18:17:52.004977 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:17:52.004944    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:18:01.351244 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.351205    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:18:01.351640 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.351409    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" containerID="cri-o://9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c" gracePeriod=30
Sep 17 18:18:01.443321 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.443287    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:18:01.443618 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.443562    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container" containerID="cri-o://16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e" gracePeriod=30
Sep 17 18:18:01.443728 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.443641    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kube-rbac-proxy" containerID="cri-o://52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a" gracePeriod=30
Sep 17 18:18:01.553326 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553290    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:18:01.553590 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553577    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kube-rbac-proxy"
Sep 17 18:18:01.553660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553592    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kube-rbac-proxy"
Sep 17 18:18:01.553660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553601    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc"
Sep 17 18:18:01.553660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553607    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc"
Sep 17 18:18:01.553660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553622    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container"
Sep 17 18:18:01.553660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553627    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container"
Sep 17 18:18:01.553924 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553668    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="097f4786-8ba4-459e-9d77-73606b4db8a8" containerName="switch-graph-0c0cc"
Sep 17 18:18:01.553924 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553675    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kserve-container"
Sep 17 18:18:01.553924 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.553682    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a1a3893-b0dc-4c9a-8055-248c3cd2f4d5" containerName="kube-rbac-proxy"
Sep 17 18:18:01.556748 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.556727    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.559855 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.559826    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-fb34c-predictor-serving-cert\""
Sep 17 18:18:01.559855 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.559844    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-fb34c-kube-rbac-proxy-sar-config\""
Sep 17 18:18:01.566644 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.566615    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:18:01.619725 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.619642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.619725 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.619695    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.619725 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.619723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.720745 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.720697    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.720745 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.720760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.721004 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.720811    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.721004 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:01.720881    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-serving-cert: secret "success-200-isvc-fb34c-predictor-serving-cert" not found
Sep 17 18:18:01.721004 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:01.720947    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls podName:73aab23b-e5e0-4fd6-9f25-c607acd58f9c nodeName:}" failed. No retries permitted until 2026-09-17 18:18:02.220929385 +0000 UTC m=+1300.644113279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls") pod "success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" (UID: "73aab23b-e5e0-4fd6-9f25-c607acd58f9c") : secret "success-200-isvc-fb34c-predictor-serving-cert" not found
Sep 17 18:18:01.721467 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.721449    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:01.730382 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:01.730347    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:02.200015 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.199982    2566 generic.go:356] "Generic (PLEG): container finished" podID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerID="52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a" exitCode=2
Sep 17 18:18:02.200175 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.200053    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerDied","Data":"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"}
Sep 17 18:18:02.224410 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.224381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:02.226813 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.226796    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") pod \"success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") " pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:02.467916 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.467813    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:02.737332 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:02.737251    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:18:02.740243 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:18:02.740218    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73aab23b_e5e0_4fd6_9f25_c607acd58f9c.slice/crio-f9254df4ddd79c013a8d1e103feb807559c312c129f6de7e984656adcf8bda2e WatchSource:0}: Error finding container f9254df4ddd79c013a8d1e103feb807559c312c129f6de7e984656adcf8bda2e: Status 404 returned error can't find the container with id f9254df4ddd79c013a8d1e103feb807559c312c129f6de7e984656adcf8bda2e
Sep 17 18:18:03.205930 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.205887    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerStarted","Data":"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"}
Sep 17 18:18:03.206101 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.205949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerStarted","Data":"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"}
Sep 17 18:18:03.206101 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.205982    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:03.206101 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.206010    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:03.206101 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.206037    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerStarted","Data":"f9254df4ddd79c013a8d1e103feb807559c312c129f6de7e984656adcf8bda2e"}
Sep 17 18:18:03.207406 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.207376    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:03.223540 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.223494    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podStartSLOduration=2.223479126 podStartE2EDuration="2.223479126s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:18:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:18:03.222348568 +0000 UTC m=+1301.645532464" watchObservedRunningTime="2026-09-17 18:18:03.223479126 +0000 UTC m=+1301.646663088"
Sep 17 18:18:03.464793 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:03.464702    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:04.209865 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.209826    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:04.693317 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.693291    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:18:04.742943 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.742910    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5458\" (UniqueName: \"kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458\") pod \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") "
Sep 17 18:18:04.742943 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.742947    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") pod \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") "
Sep 17 18:18:04.743175 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.742985    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config\") pod \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\" (UID: \"f6476fb1-0c87-4d63-9dea-a69e622d1d38\") "
Sep 17 18:18:04.743419 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.743386    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f6476fb1-0c87-4d63-9dea-a69e622d1d38" (UID: "f6476fb1-0c87-4d63-9dea-a69e622d1d38"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:18:04.745147 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.745116    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f6476fb1-0c87-4d63-9dea-a69e622d1d38" (UID: "f6476fb1-0c87-4d63-9dea-a69e622d1d38"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:18:04.745236 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.745150    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458" (OuterVolumeSpecName: "kube-api-access-p5458") pod "f6476fb1-0c87-4d63-9dea-a69e622d1d38" (UID: "f6476fb1-0c87-4d63-9dea-a69e622d1d38"). InnerVolumeSpecName "kube-api-access-p5458". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:18:04.844630 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.844545    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5458\" (UniqueName: \"kubernetes.io/projected/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-api-access-p5458\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:04.844630 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.844573    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f6476fb1-0c87-4d63-9dea-a69e622d1d38-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:04.844630 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.844583    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f6476fb1-0c87-4d63-9dea-a69e622d1d38-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:04.871375 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871341    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:04.871645 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871633    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container"
Sep 17 18:18:04.871682 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871647    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container"
Sep 17 18:18:04.871682 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871655    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kube-rbac-proxy"
Sep 17 18:18:04.871682 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871660    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kube-rbac-proxy"
Sep 17 18:18:04.871803 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871702    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kserve-container"
Sep 17 18:18:04.871803 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.871712    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerName="kube-rbac-proxy"
Sep 17 18:18:04.874517 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.874502    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:04.877168 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.877140    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-b0976-kube-rbac-proxy-sar-config\""
Sep 17 18:18:04.877168 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.877155    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-b0976-serving-cert\""
Sep 17 18:18:04.882156 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.882126    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:04.945893 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.945856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:04.945893 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:04.945895    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.046446 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.046409    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.046590 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.046452    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.046590 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:05.046556    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-b0976-serving-cert: secret "ensemble-graph-b0976-serving-cert" not found
Sep 17 18:18:05.046669 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:05.046632    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls podName:79aac34f-8028-4c1a-bfb6-6ababbac4563 nodeName:}" failed. No retries permitted until 2026-09-17 18:18:05.546607721 +0000 UTC m=+1303.969791594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls") pod "ensemble-graph-b0976-8d4c9ffcc-vg6pl" (UID: "79aac34f-8028-4c1a-bfb6-6ababbac4563") : secret "ensemble-graph-b0976-serving-cert" not found
Sep 17 18:18:05.047122 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.047105    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.215193 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.215108    2566 generic.go:356] "Generic (PLEG): container finished" podID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" containerID="16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e" exitCode=0
Sep 17 18:18:05.215193 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.215178    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"
Sep 17 18:18:05.215623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.215194    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerDied","Data":"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"}
Sep 17 18:18:05.215623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.215266    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm" event={"ID":"f6476fb1-0c87-4d63-9dea-a69e622d1d38","Type":"ContainerDied","Data":"ff46d10f8ee84ea5d28c4f8e65cc031f5cf9f5beb820890f3eef307be2805746"}
Sep 17 18:18:05.215623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.215304    2566 scope.go:117] "RemoveContainer" containerID="52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"
Sep 17 18:18:05.223980 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.223961    2566 scope.go:117] "RemoveContainer" containerID="16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"
Sep 17 18:18:05.231302 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.231284    2566 scope.go:117] "RemoveContainer" containerID="52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"
Sep 17 18:18:05.231585 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:05.231562    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a\": container with ID starting with 52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a not found: ID does not exist" containerID="52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"
Sep 17 18:18:05.231723 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.231590    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a"} err="failed to get container status \"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a\": rpc error: code = NotFound desc = could not find container \"52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a\": container with ID starting with 52b13340dbc99edc8c49b2039b02d0e87a426e9fd0226d51c587de4fb970db2a not found: ID does not exist"
Sep 17 18:18:05.231723 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.231609    2566 scope.go:117] "RemoveContainer" containerID="16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"
Sep 17 18:18:05.232269 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:05.232252    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e\": container with ID starting with 16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e not found: ID does not exist" containerID="16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"
Sep 17 18:18:05.232314 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.232275    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e"} err="failed to get container status \"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e\": rpc error: code = NotFound desc = could not find container \"16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e\": container with ID starting with 16ea1ec709f8f1eb89ef56ae04dee48d400629066099506e27935f9d84cefd8e not found: ID does not exist"
Sep 17 18:18:05.236166 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.236144    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:18:05.239675 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.239654    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4a77e-predictor-5b6b79bb5f-j8rlm"]
Sep 17 18:18:05.550464 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.550434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.552993 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.552963    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") pod \"ensemble-graph-b0976-8d4c9ffcc-vg6pl\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") " pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:05.653140 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.653103    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6476fb1-0c87-4d63-9dea-a69e622d1d38" path="/var/lib/kubelet/pods/f6476fb1-0c87-4d63-9dea-a69e622d1d38/volumes"
Sep 17 18:18:05.784765 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:05.784719    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:06.054407 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:06.054382    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:06.056879 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:18:06.056853    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79aac34f_8028_4c1a_bfb6_6ababbac4563.slice/crio-3affc8889b0f46300cffb4076cdb2a4cfef25477bcc972de0fb03f02423a8adf WatchSource:0}: Error finding container 3affc8889b0f46300cffb4076cdb2a4cfef25477bcc972de0fb03f02423a8adf: Status 404 returned error can't find the container with id 3affc8889b0f46300cffb4076cdb2a4cfef25477bcc972de0fb03f02423a8adf
Sep 17 18:18:06.220139 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:06.220107    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" event={"ID":"79aac34f-8028-4c1a-bfb6-6ababbac4563","Type":"ContainerStarted","Data":"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"}
Sep 17 18:18:06.220552 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:06.220165    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" event={"ID":"79aac34f-8028-4c1a-bfb6-6ababbac4563","Type":"ContainerStarted","Data":"3affc8889b0f46300cffb4076cdb2a4cfef25477bcc972de0fb03f02423a8adf"}
Sep 17 18:18:06.220552 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:06.220206    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:06.236675 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:06.236632    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podStartSLOduration=2.23661566 podStartE2EDuration="2.23661566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:18:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:18:06.235154023 +0000 UTC m=+1304.658337918" watchObservedRunningTime="2026-09-17 18:18:06.23661566 +0000 UTC m=+1304.659799555"
Sep 17 18:18:08.464729 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:08.464693    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:09.214219 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:09.214193    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:09.214602 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:09.214578    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:12.230010 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:12.229980    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:13.464343 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:13.464303    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:13.464730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:13.464410    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:18:14.936212 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:14.936133    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:14.936578 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:14.936331    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" containerID="cri-o://009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c" gracePeriod=30
Sep 17 18:18:15.037840 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.037807    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:18:15.038190 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.038038    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container" containerID="cri-o://1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249" gracePeriod=30
Sep 17 18:18:15.038190 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.038147    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kube-rbac-proxy" containerID="cri-o://b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb" gracePeriod=30
Sep 17 18:18:15.157697 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.157669    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:18:15.160753 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.160738    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.163359 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.163341    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2aa9e-predictor-serving-cert\""
Sep 17 18:18:15.163463 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.163400    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-2aa9e-kube-rbac-proxy-sar-config\""
Sep 17 18:18:15.174606 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.174584    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:18:15.227007 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.226931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.227007 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.226970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.227007 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.226992    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w5gw\" (UniqueName: \"kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.256958 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.256927    2566 generic.go:356] "Generic (PLEG): container finished" podID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerID="b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb" exitCode=2
Sep 17 18:18:15.257111 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.257006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerDied","Data":"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"}
Sep 17 18:18:15.328110 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.328071    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.328273 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.328123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.328273 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.328152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w5gw\" (UniqueName: \"kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.328877 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.328859    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.330531 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.330514    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.336919 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.336899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w5gw\" (UniqueName: \"kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw\") pod \"success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") " pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.471365 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.471320    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:15.743451 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:15.743420    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:18:15.747739 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:18:15.747708    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c1a4099_40df_45b1_8eda_fb25817123ba.slice/crio-3dc06eed6af4a095473f188a1db586d6e4a1a5b1667ed3b1a24606b96cc6a99a WatchSource:0}: Error finding container 3dc06eed6af4a095473f188a1db586d6e4a1a5b1667ed3b1a24606b96cc6a99a: Status 404 returned error can't find the container with id 3dc06eed6af4a095473f188a1db586d6e4a1a5b1667ed3b1a24606b96cc6a99a
Sep 17 18:18:16.262543 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.262502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerStarted","Data":"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"}
Sep 17 18:18:16.262961 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.262577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerStarted","Data":"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"}
Sep 17 18:18:16.262961 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.262719    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:16.262961 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.262795    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:16.262961 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.262830    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerStarted","Data":"3dc06eed6af4a095473f188a1db586d6e4a1a5b1667ed3b1a24606b96cc6a99a"}
Sep 17 18:18:16.264309 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.264286    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:18:16.280634 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.280590    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podStartSLOduration=1.280575279 podStartE2EDuration="1.280575279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:18:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:18:16.279668853 +0000 UTC m=+1314.702852781" watchObservedRunningTime="2026-09-17 18:18:16.280575279 +0000 UTC m=+1314.703759174"
Sep 17 18:18:16.999159 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:16.999116    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.23:8643/healthz\": dial tcp 10.132.0.23:8643: connect: connection refused"
Sep 17 18:18:17.229411 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:17.229369    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:17.266501 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:17.266424    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:18:18.464619 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.464584    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:18.481967 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.481942    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:18:18.555841 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.555808    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config\") pod \"e85e55d9-1383-4434-b8e6-63aedad2ab61\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") "
Sep 17 18:18:18.556011 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.555876    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2lxd\" (UniqueName: \"kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd\") pod \"e85e55d9-1383-4434-b8e6-63aedad2ab61\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") "
Sep 17 18:18:18.556011 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.555900    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls\") pod \"e85e55d9-1383-4434-b8e6-63aedad2ab61\" (UID: \"e85e55d9-1383-4434-b8e6-63aedad2ab61\") "
Sep 17 18:18:18.556124 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.556034    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e85e55d9-1383-4434-b8e6-63aedad2ab61" (UID: "e85e55d9-1383-4434-b8e6-63aedad2ab61"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:18:18.556124 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.556106    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:18.557934 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.557910    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd" (OuterVolumeSpecName: "kube-api-access-f2lxd") pod "e85e55d9-1383-4434-b8e6-63aedad2ab61" (UID: "e85e55d9-1383-4434-b8e6-63aedad2ab61"). InnerVolumeSpecName "kube-api-access-f2lxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:18:18.558028 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.557913    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e85e55d9-1383-4434-b8e6-63aedad2ab61" (UID: "e85e55d9-1383-4434-b8e6-63aedad2ab61"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:18:18.657235 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.657194    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2lxd\" (UniqueName: \"kubernetes.io/projected/e85e55d9-1383-4434-b8e6-63aedad2ab61-kube-api-access-f2lxd\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:18.657235 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:18.657232    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e85e55d9-1383-4434-b8e6-63aedad2ab61-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:19.215528 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.215490    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:19.275532 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.275497    2566 generic.go:356] "Generic (PLEG): container finished" podID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerID="1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249" exitCode=0
Sep 17 18:18:19.275730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.275545    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"
Sep 17 18:18:19.275730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.275583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerDied","Data":"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"}
Sep 17 18:18:19.275730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.275650    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm" event={"ID":"e85e55d9-1383-4434-b8e6-63aedad2ab61","Type":"ContainerDied","Data":"cb6f62b7f8119c05cf68d65be0062e9ec68fb5d513a4c2a2faf6fc839237e875"}
Sep 17 18:18:19.275730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.275692    2566 scope.go:117] "RemoveContainer" containerID="b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"
Sep 17 18:18:19.283755 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.283736    2566 scope.go:117] "RemoveContainer" containerID="1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"
Sep 17 18:18:19.291089 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.291074    2566 scope.go:117] "RemoveContainer" containerID="b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"
Sep 17 18:18:19.291353 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:19.291335    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb\": container with ID starting with b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb not found: ID does not exist" containerID="b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"
Sep 17 18:18:19.291400 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.291361    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb"} err="failed to get container status \"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb\": rpc error: code = NotFound desc = could not find container \"b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb\": container with ID starting with b6d839b2c77b113de2e9c0f8ac9d0489173c05a9e2713d7de2f179d36a8483fb not found: ID does not exist"
Sep 17 18:18:19.291400 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.291377    2566 scope.go:117] "RemoveContainer" containerID="1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"
Sep 17 18:18:19.291619 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:19.291601    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249\": container with ID starting with 1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249 not found: ID does not exist" containerID="1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"
Sep 17 18:18:19.291678 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.291629    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249"} err="failed to get container status \"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249\": rpc error: code = NotFound desc = could not find container \"1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249\": container with ID starting with 1f28e6b823b4db0a0c14df18f10cc5ec348109e73f1bce81f039bb384aeaa249 not found: ID does not exist"
Sep 17 18:18:19.296300 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.296277    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:18:19.301410 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.301390    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-b0976-predictor-6c4d57ffcf-kxldm"]
Sep 17 18:18:19.657815 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:19.655163    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" path="/var/lib/kubelet/pods/e85e55d9-1383-4434-b8e6-63aedad2ab61/volumes"
Sep 17 18:18:22.229367 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:22.229328    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:22.271333 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:22.271306    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:18:22.271717 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:22.271691    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:18:23.465173 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:23.465134    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:27.229332 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:27.229286    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:27.229762 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:27.229396    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:28.464337 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:28.464298    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:29.215260 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:29.215218    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:31.494941 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.494916    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:18:31.655206 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.655124    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls\") pod \"8ba66a72-ae61-430f-98a1-1b93315a22cb\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") "
Sep 17 18:18:31.655348 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.655243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle\") pod \"8ba66a72-ae61-430f-98a1-1b93315a22cb\" (UID: \"8ba66a72-ae61-430f-98a1-1b93315a22cb\") "
Sep 17 18:18:31.655594 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.655569    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8ba66a72-ae61-430f-98a1-1b93315a22cb" (UID: "8ba66a72-ae61-430f-98a1-1b93315a22cb"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:18:31.657172 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.657153    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8ba66a72-ae61-430f-98a1-1b93315a22cb" (UID: "8ba66a72-ae61-430f-98a1-1b93315a22cb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:18:31.756702 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.756668    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ba66a72-ae61-430f-98a1-1b93315a22cb-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:31.756702 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:31.756695    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8ba66a72-ae61-430f-98a1-1b93315a22cb-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:32.228805 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.228742    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:32.272467 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.272424    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:18:32.323986 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.323947    2566 generic.go:356] "Generic (PLEG): container finished" podID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerID="9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c" exitCode=0
Sep 17 18:18:32.324179 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.323996    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"
Sep 17 18:18:32.324179 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.324001    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" event={"ID":"8ba66a72-ae61-430f-98a1-1b93315a22cb","Type":"ContainerDied","Data":"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"}
Sep 17 18:18:32.324179 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.324120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp" event={"ID":"8ba66a72-ae61-430f-98a1-1b93315a22cb","Type":"ContainerDied","Data":"82873101d8779f5b78f9c746c362473c1ba5832f46f48be60aee46050f609957"}
Sep 17 18:18:32.324179 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.324153    2566 scope.go:117] "RemoveContainer" containerID="9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"
Sep 17 18:18:32.332568 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.332544    2566 scope.go:117] "RemoveContainer" containerID="9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"
Sep 17 18:18:32.332938 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:32.332920    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c\": container with ID starting with 9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c not found: ID does not exist" containerID="9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"
Sep 17 18:18:32.333008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.332945    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c"} err="failed to get container status \"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c\": rpc error: code = NotFound desc = could not find container \"9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c\": container with ID starting with 9f2bffe3193920897602a06798145db429ec610d422843a3ddd92a0649f03a3c not found: ID does not exist"
Sep 17 18:18:32.344029 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.344000    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:18:32.345918 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:32.345893    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4a77e-7897f464d9-9scnp"]
Sep 17 18:18:33.652737 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:33.652704    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" path="/var/lib/kubelet/pods/8ba66a72-ae61-430f-98a1-1b93315a22cb/volumes"
Sep 17 18:18:37.228768 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:37.228680    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:39.215037 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:39.215004    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 17 18:18:42.228983 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:42.228944    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:18:42.271757 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:42.271719    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:18:45.118226 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.118199    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:45.256183 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.256108    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") pod \"79aac34f-8028-4c1a-bfb6-6ababbac4563\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") "
Sep 17 18:18:45.256183 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.256162    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle\") pod \"79aac34f-8028-4c1a-bfb6-6ababbac4563\" (UID: \"79aac34f-8028-4c1a-bfb6-6ababbac4563\") "
Sep 17 18:18:45.256456 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.256432    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "79aac34f-8028-4c1a-bfb6-6ababbac4563" (UID: "79aac34f-8028-4c1a-bfb6-6ababbac4563"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:18:45.258109 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.258084    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "79aac34f-8028-4c1a-bfb6-6ababbac4563" (UID: "79aac34f-8028-4c1a-bfb6-6ababbac4563"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:18:45.356571 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.356540    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/79aac34f-8028-4c1a-bfb6-6ababbac4563-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:45.356571 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.356566    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79aac34f-8028-4c1a-bfb6-6ababbac4563-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:18:45.378206 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.378176    2566 generic.go:356] "Generic (PLEG): container finished" podID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerID="009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c" exitCode=0
Sep 17 18:18:45.378342 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.378223    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"
Sep 17 18:18:45.378342 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.378225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" event={"ID":"79aac34f-8028-4c1a-bfb6-6ababbac4563","Type":"ContainerDied","Data":"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"}
Sep 17 18:18:45.378453 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.378364    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl" event={"ID":"79aac34f-8028-4c1a-bfb6-6ababbac4563","Type":"ContainerDied","Data":"3affc8889b0f46300cffb4076cdb2a4cfef25477bcc972de0fb03f02423a8adf"}
Sep 17 18:18:45.378453 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.378406    2566 scope.go:117] "RemoveContainer" containerID="009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"
Sep 17 18:18:45.387195 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.387179    2566 scope.go:117] "RemoveContainer" containerID="009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"
Sep 17 18:18:45.387462 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:18:45.387445    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c\": container with ID starting with 009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c not found: ID does not exist" containerID="009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"
Sep 17 18:18:45.387509 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.387473    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c"} err="failed to get container status \"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c\": rpc error: code = NotFound desc = could not find container \"009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c\": container with ID starting with 009496c48c6e174c3993060f31ccf055afd5da9c8a98f31f6f060d48c4890e2c not found: ID does not exist"
Sep 17 18:18:45.397997 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.397972    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:45.401239 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.401214    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0976-8d4c9ffcc-vg6pl"]
Sep 17 18:18:45.653073 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:45.653040    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" path="/var/lib/kubelet/pods/79aac34f-8028-4c1a-bfb6-6ababbac4563/volumes"
Sep 17 18:18:49.215951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:49.215923    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:18:52.272393 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:18:52.272354    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:19:01.543758 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.543719    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544114    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kube-rbac-proxy"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544131    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kube-rbac-proxy"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544148    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544156    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544165    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544173    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544182    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544190    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544269    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kserve-container"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544282    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="79aac34f-8028-4c1a-bfb6-6ababbac4563" containerName="ensemble-graph-b0976"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544291    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e85e55d9-1383-4434-b8e6-63aedad2ab61" containerName="kube-rbac-proxy"
Sep 17 18:19:01.544329 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.544300    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ba66a72-ae61-430f-98a1-1b93315a22cb" containerName="sequence-graph-4a77e"
Sep 17 18:19:01.548537 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.548518    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.551474 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.551451    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fb34c-serving-cert\""
Sep 17 18:19:01.551581 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.551477    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fb34c-kube-rbac-proxy-sar-config\""
Sep 17 18:19:01.557120 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.557098    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:01.574729 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.574702    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.574869 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.574747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.675516 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.675483    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.675690 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.675538    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.676711 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.676677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.681055 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.678877    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls\") pod \"sequence-graph-fb34c-7887db4c77-7fjcb\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") " pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:01.859360 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:01.859269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:02.127427 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.127270    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:02.130370 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:19:02.130344    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e6bb9c6_e964_4def_bec3_21b5f57a9265.slice/crio-dadfa7d13c875e0dfc0ca54a5581bca77125d330e081f56788812a0dfd0e2c44 WatchSource:0}: Error finding container dadfa7d13c875e0dfc0ca54a5581bca77125d330e081f56788812a0dfd0e2c44: Status 404 returned error can't find the container with id dadfa7d13c875e0dfc0ca54a5581bca77125d330e081f56788812a0dfd0e2c44
Sep 17 18:19:02.273152 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.273124    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:19:02.439391 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.439311    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" event={"ID":"4e6bb9c6-e964-4def-bec3-21b5f57a9265","Type":"ContainerStarted","Data":"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"}
Sep 17 18:19:02.439391 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.439359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" event={"ID":"4e6bb9c6-e964-4def-bec3-21b5f57a9265","Type":"ContainerStarted","Data":"dadfa7d13c875e0dfc0ca54a5581bca77125d330e081f56788812a0dfd0e2c44"}
Sep 17 18:19:02.439601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.439435    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:02.455142 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:02.455092    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podStartSLOduration=1.455075028 podStartE2EDuration="1.455075028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:19:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:19:02.454585264 +0000 UTC m=+1360.877769158" watchObservedRunningTime="2026-09-17 18:19:02.455075028 +0000 UTC m=+1360.878258923"
Sep 17 18:19:08.448733 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:08.448697    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:11.624013 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.623979    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:11.624528 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.624158    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" containerID="cri-o://f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26" gracePeriod=30
Sep 17 18:19:11.713464 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.713433    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:19:11.713695 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.713654    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container" containerID="cri-o://76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5" gracePeriod=30
Sep 17 18:19:11.713811 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.713703    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kube-rbac-proxy" containerID="cri-o://f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed" gracePeriod=30
Sep 17 18:19:11.836013 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.835984    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:19:11.838943 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.838928    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.842042 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.842023    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-04b70-predictor-serving-cert\""
Sep 17 18:19:11.842171 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.842023    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-04b70-kube-rbac-proxy-sar-config\""
Sep 17 18:19:11.844770 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.844749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.844881 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.844776    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.844881 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.844823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzxvj\" (UniqueName: \"kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.849356 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.849335    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:19:11.946034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.945944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.946034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.945978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.946034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.946005    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzxvj\" (UniqueName: \"kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.946582 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.946560    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.948444 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.948422    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:11.958268 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:11.958244    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzxvj\" (UniqueName: \"kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj\") pod \"success-200-isvc-04b70-predictor-697dcfdbb9-k58nf\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") " pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:12.148167 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:12.148119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:12.418464 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:12.418436    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:19:12.420757 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:19:12.420731    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d45d1ae_0e56_4f48_98d9_79211d1757eb.slice/crio-0433fcd6372a1369b69e8a74d9d84d834f54dc93ddafc63c32adef1dc3b5f0d8 WatchSource:0}: Error finding container 0433fcd6372a1369b69e8a74d9d84d834f54dc93ddafc63c32adef1dc3b5f0d8: Status 404 returned error can't find the container with id 0433fcd6372a1369b69e8a74d9d84d834f54dc93ddafc63c32adef1dc3b5f0d8
Sep 17 18:19:12.480084 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:12.480059    2566 generic.go:356] "Generic (PLEG): container finished" podID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerID="f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed" exitCode=2
Sep 17 18:19:12.480191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:12.480130    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerDied","Data":"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"}
Sep 17 18:19:12.481763 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:12.481742    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerStarted","Data":"0433fcd6372a1369b69e8a74d9d84d834f54dc93ddafc63c32adef1dc3b5f0d8"}
Sep 17 18:19:13.448191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.448136    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:13.487230 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.487192    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerStarted","Data":"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"}
Sep 17 18:19:13.487427 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.487248    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerStarted","Data":"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"}
Sep 17 18:19:13.487427 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.487275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:13.487427 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.487304    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:13.488563 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.488540    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:13.505392 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:13.505341    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podStartSLOduration=2.505322624 podStartE2EDuration="2.505322624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:19:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:19:13.503995572 +0000 UTC m=+1371.927179466" watchObservedRunningTime="2026-09-17 18:19:13.505322624 +0000 UTC m=+1371.928506518"
Sep 17 18:19:14.211077 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.211028    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": dial tcp 10.132.0.24:8643: connect: connection refused"
Sep 17 18:19:14.491514 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.491412    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:14.963155 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.963128    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:19:14.969624 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.969602    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config\") pod \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") "
Sep 17 18:19:14.969763 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.969674    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt\") pod \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") "
Sep 17 18:19:14.969763 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.969700    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") pod \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\" (UID: \"73aab23b-e5e0-4fd6-9f25-c607acd58f9c\") "
Sep 17 18:19:14.970019 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.969989    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "73aab23b-e5e0-4fd6-9f25-c607acd58f9c" (UID: "73aab23b-e5e0-4fd6-9f25-c607acd58f9c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:19:14.971634 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.971609    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt" (OuterVolumeSpecName: "kube-api-access-zn7dt") pod "73aab23b-e5e0-4fd6-9f25-c607acd58f9c" (UID: "73aab23b-e5e0-4fd6-9f25-c607acd58f9c"). InnerVolumeSpecName "kube-api-access-zn7dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:19:14.971709 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:14.971675    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "73aab23b-e5e0-4fd6-9f25-c607acd58f9c" (UID: "73aab23b-e5e0-4fd6-9f25-c607acd58f9c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:19:15.070355 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.070323    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zn7dt\" (UniqueName: \"kubernetes.io/projected/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-api-access-zn7dt\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:19:15.070355 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.070354    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:19:15.070559 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.070364    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/73aab23b-e5e0-4fd6-9f25-c607acd58f9c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:19:15.153822 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.153774    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:19:15.154072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154061    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container"
Sep 17 18:19:15.154113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154074    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container"
Sep 17 18:19:15.154113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154093    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kube-rbac-proxy"
Sep 17 18:19:15.154113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154098    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kube-rbac-proxy"
Sep 17 18:19:15.154198 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154143    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kserve-container"
Sep 17 18:19:15.154198 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.154153    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerName="kube-rbac-proxy"
Sep 17 18:19:15.156948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.156933    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.159490 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.159468    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2aa9e-serving-cert\""
Sep 17 18:19:15.159614 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.159600    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-2aa9e-kube-rbac-proxy-sar-config\""
Sep 17 18:19:15.165374 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.165351    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:19:15.170998 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.170977    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.171096 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.171011    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.272072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.272031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.272239 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.272088    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.272805 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.272758    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.274544 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.274524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls\") pod \"ensemble-graph-2aa9e-744c775558-zn5kc\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") " pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.467143 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.467053    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:15.497232 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.497203    2566 generic.go:356] "Generic (PLEG): container finished" podID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" containerID="76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5" exitCode=0
Sep 17 18:19:15.497575 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.497250    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"
Sep 17 18:19:15.497575 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.497295    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerDied","Data":"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"}
Sep 17 18:19:15.497575 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.497374    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf" event={"ID":"73aab23b-e5e0-4fd6-9f25-c607acd58f9c","Type":"ContainerDied","Data":"f9254df4ddd79c013a8d1e103feb807559c312c129f6de7e984656adcf8bda2e"}
Sep 17 18:19:15.497575 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.497415    2566 scope.go:117] "RemoveContainer" containerID="f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"
Sep 17 18:19:15.506915 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.506898    2566 scope.go:117] "RemoveContainer" containerID="76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"
Sep 17 18:19:15.514277 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.514242    2566 scope.go:117] "RemoveContainer" containerID="f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"
Sep 17 18:19:15.514605 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:19:15.514579    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed\": container with ID starting with f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed not found: ID does not exist" containerID="f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"
Sep 17 18:19:15.514716 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.514621    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed"} err="failed to get container status \"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed\": rpc error: code = NotFound desc = could not find container \"f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed\": container with ID starting with f777c5e9b6eccb828e9c0d64a344965c8ef0e44396b0b0d123d97679ff740fed not found: ID does not exist"
Sep 17 18:19:15.514716 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.514644    2566 scope.go:117] "RemoveContainer" containerID="76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"
Sep 17 18:19:15.515213 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:19:15.515170    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5\": container with ID starting with 76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5 not found: ID does not exist" containerID="76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"
Sep 17 18:19:15.515213 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.515200    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5"} err="failed to get container status \"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5\": rpc error: code = NotFound desc = could not find container \"76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5\": container with ID starting with 76bcd6eadb846d954d481223aecfb997daeacb8497873b718b43948acd9c0fd5 not found: ID does not exist"
Sep 17 18:19:15.520871 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.520846    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:19:15.523325 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.523301    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-fb34c-predictor-64f6cd7f8c-d9vcf"]
Sep 17 18:19:15.652704 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.652667    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73aab23b-e5e0-4fd6-9f25-c607acd58f9c" path="/var/lib/kubelet/pods/73aab23b-e5e0-4fd6-9f25-c607acd58f9c/volumes"
Sep 17 18:19:15.739387 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:15.739318    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:19:15.742486 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:19:15.742462    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6220a9c_5fa3_499b_aa5b_46243ff5b9a0.slice/crio-df0bda330b949b657cc0e15475014616ead23c726249dc0f6d060680e47a1d2b WatchSource:0}: Error finding container df0bda330b949b657cc0e15475014616ead23c726249dc0f6d060680e47a1d2b: Status 404 returned error can't find the container with id df0bda330b949b657cc0e15475014616ead23c726249dc0f6d060680e47a1d2b
Sep 17 18:19:16.503302 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:16.503266    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" event={"ID":"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0","Type":"ContainerStarted","Data":"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"}
Sep 17 18:19:16.503730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:16.503325    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" event={"ID":"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0","Type":"ContainerStarted","Data":"df0bda330b949b657cc0e15475014616ead23c726249dc0f6d060680e47a1d2b"}
Sep 17 18:19:16.503730 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:16.503356    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:16.520190 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:16.520143    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podStartSLOduration=1.5201278 podStartE2EDuration="1.5201278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:19:16.518507011 +0000 UTC m=+1374.941690905" watchObservedRunningTime="2026-09-17 18:19:16.5201278 +0000 UTC m=+1374.943311694"
Sep 17 18:19:18.447881 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:18.447844    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:19.496129 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:19.496103    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:19:19.496567 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:19.496545    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:22.513580 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:22.513552    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:19:23.447385 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:23.447341    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:23.447598 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:23.447443    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:28.447970 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:28.447920    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:29.496762 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:29.496725    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:33.447225 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:33.447179    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:38.447517 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:38.447466    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:19:39.497072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:39.497035    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:41.785480 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.785453    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:41.874994 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.874944    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls\") pod \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") "
Sep 17 18:19:41.874994 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.875008    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle\") pod \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\" (UID: \"4e6bb9c6-e964-4def-bec3-21b5f57a9265\") "
Sep 17 18:19:41.875394 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.875362    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "4e6bb9c6-e964-4def-bec3-21b5f57a9265" (UID: "4e6bb9c6-e964-4def-bec3-21b5f57a9265"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:19:41.877046 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.877024    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4e6bb9c6-e964-4def-bec3-21b5f57a9265" (UID: "4e6bb9c6-e964-4def-bec3-21b5f57a9265"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:19:41.976093 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.976055    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4e6bb9c6-e964-4def-bec3-21b5f57a9265-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:19:41.976093 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:41.976087    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4e6bb9c6-e964-4def-bec3-21b5f57a9265-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:19:42.605125 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.605091    2566 generic.go:356] "Generic (PLEG): container finished" podID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerID="f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26" exitCode=137
Sep 17 18:19:42.605125 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.605131    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" event={"ID":"4e6bb9c6-e964-4def-bec3-21b5f57a9265","Type":"ContainerDied","Data":"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"}
Sep 17 18:19:42.605327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.605136    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"
Sep 17 18:19:42.605327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.605203    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb" event={"ID":"4e6bb9c6-e964-4def-bec3-21b5f57a9265","Type":"ContainerDied","Data":"dadfa7d13c875e0dfc0ca54a5581bca77125d330e081f56788812a0dfd0e2c44"}
Sep 17 18:19:42.605327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.605227    2566 scope.go:117] "RemoveContainer" containerID="f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"
Sep 17 18:19:42.613595 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.613579    2566 scope.go:117] "RemoveContainer" containerID="f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"
Sep 17 18:19:42.613863 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:19:42.613842    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26\": container with ID starting with f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26 not found: ID does not exist" containerID="f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"
Sep 17 18:19:42.613944 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.613869    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26"} err="failed to get container status \"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26\": rpc error: code = NotFound desc = could not find container \"f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26\": container with ID starting with f491c157d188edb7df57084cceb7012cbe3a8ca0c07c422b04a17bfb62ea7b26 not found: ID does not exist"
Sep 17 18:19:42.626071 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.626043    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:42.631078 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:42.631057    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fb34c-7887db4c77-7fjcb"]
Sep 17 18:19:43.652645 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:43.652613    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" path="/var/lib/kubelet/pods/4e6bb9c6-e964-4def-bec3-21b5f57a9265/volumes"
Sep 17 18:19:49.496540 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:49.496502    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:19:59.496938 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:19:59.496902    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:20:11.857398 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.857321    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:20:11.857869 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.857600    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c"
Sep 17 18:20:11.857869 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.857610    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c"
Sep 17 18:20:11.857869 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.857653    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e6bb9c6-e964-4def-bec3-21b5f57a9265" containerName="sequence-graph-fb34c"
Sep 17 18:20:11.860757 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.860741    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:11.863435 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.863416    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-04b70-kube-rbac-proxy-sar-config\""
Sep 17 18:20:11.863557 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.863501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-04b70-serving-cert\""
Sep 17 18:20:11.868707 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:11.868685    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:20:12.002418 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.002382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.002599 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.002450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.103271 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.103232    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.103442 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.103287    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.103902 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.103883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.105594 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.105576    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls\") pod \"sequence-graph-04b70-6564cd7dcf-j6jhv\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") " pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.171909 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.171821    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.438815 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.438766    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:20:12.442672 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:20:12.442634    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b0d4577_7276_4745_a654_989995e8d7ac.slice/crio-79a16311ac102ca355e70db90216542493ff99d8a8729819e36f7eb07401f0dd WatchSource:0}: Error finding container 79a16311ac102ca355e70db90216542493ff99d8a8729819e36f7eb07401f0dd: Status 404 returned error can't find the container with id 79a16311ac102ca355e70db90216542493ff99d8a8729819e36f7eb07401f0dd
Sep 17 18:20:12.712892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.712795    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" event={"ID":"3b0d4577-7276-4745-a654-989995e8d7ac","Type":"ContainerStarted","Data":"4c37cf7da9be5ae2dfd4f41eefa6a5bb77929fc6ea06f29c2320977679e9df67"}
Sep 17 18:20:12.712892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.712847    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" event={"ID":"3b0d4577-7276-4745-a654-989995e8d7ac","Type":"ContainerStarted","Data":"79a16311ac102ca355e70db90216542493ff99d8a8729819e36f7eb07401f0dd"}
Sep 17 18:20:12.713111 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.712939    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:20:12.729277 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:12.729230    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podStartSLOduration=1.729213944 podStartE2EDuration="1.729213944s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:20:12.727633543 +0000 UTC m=+1431.150817439" watchObservedRunningTime="2026-09-17 18:20:12.729213944 +0000 UTC m=+1431.152397839"
Sep 17 18:20:18.722483 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:20:18.722453    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:27:29.754536 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.754502    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:27:29.755050 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.754720    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" containerID="cri-o://94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0" gracePeriod=30
Sep 17 18:27:29.867442 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.867408    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:27:29.867741 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.867690    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" containerID="cri-o://7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1" gracePeriod=30
Sep 17 18:27:29.867867 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.867758    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kube-rbac-proxy" containerID="cri-o://6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92" gracePeriod=30
Sep 17 18:27:29.965458 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.965427    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:27:29.968673 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.968656    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:29.971316 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.971299    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e0a3a-predictor-serving-cert\""
Sep 17 18:27:29.971512 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.971483    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-e0a3a-kube-rbac-proxy-sar-config\""
Sep 17 18:27:29.979845 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:29.979824    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:27:30.084344 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.084306    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.084514 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.084353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.084514 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.084422    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cjzv\" (UniqueName: \"kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.185263 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.185222    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cjzv\" (UniqueName: \"kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.185458 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.185271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.185458 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.185294    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.185458 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:27:30.185412    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-serving-cert: secret "success-200-isvc-e0a3a-predictor-serving-cert" not found
Sep 17 18:27:30.185621 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:27:30.185495    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls podName:bd07cc66-7379-42e7-972f-11236706b28d nodeName:}" failed. No retries permitted until 2026-09-17 18:27:30.685463091 +0000 UTC m=+1869.108646964 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls") pod "success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" (UID: "bd07cc66-7379-42e7-972f-11236706b28d") : secret "success-200-isvc-e0a3a-predictor-serving-cert" not found
Sep 17 18:27:30.185918 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.185898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.194051 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.194021    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cjzv\" (UniqueName: \"kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.258776 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.258745    2566 generic.go:356] "Generic (PLEG): container finished" podID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerID="6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92" exitCode=2
Sep 17 18:27:30.258956 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.258825    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerDied","Data":"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"}
Sep 17 18:27:30.689727 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.689680    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.692088 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.692068    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") pod \"success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") " pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:30.878998 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:30.878962    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:31.148472 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:31.148447    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:27:31.151037 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:27:31.150998    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd07cc66_7379_42e7_972f_11236706b28d.slice/crio-45d64cee60c7598bdb358749254172ffc5989d28b05173d44159c73d6eead443 WatchSource:0}: Error finding container 45d64cee60c7598bdb358749254172ffc5989d28b05173d44159c73d6eead443: Status 404 returned error can't find the container with id 45d64cee60c7598bdb358749254172ffc5989d28b05173d44159c73d6eead443
Sep 17 18:27:31.263941 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:31.263912    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerStarted","Data":"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"}
Sep 17 18:27:31.264030 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:31.263981    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerStarted","Data":"45d64cee60c7598bdb358749254172ffc5989d28b05173d44159c73d6eead443"}
Sep 17 18:27:32.266686 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.266647    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.26:8643/healthz\": dial tcp 10.132.0.26:8643: connect: connection refused"
Sep 17 18:27:32.269064 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.269035    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerStarted","Data":"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"}
Sep 17 18:27:32.269202 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.269174    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:32.272090 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.272064    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.26:8080: connect: connection refused"
Sep 17 18:27:32.285885 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.285845    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podStartSLOduration=3.285831002 podStartE2EDuration="3.285831002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:27:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:27:32.284773497 +0000 UTC m=+1870.707957393" watchObservedRunningTime="2026-09-17 18:27:32.285831002 +0000 UTC m=+1870.709014896"
Sep 17 18:27:32.512319 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.512276    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:32.908394 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:32.908366    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:27:33.007927 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.007846    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w5gw\" (UniqueName: \"kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw\") pod \"5c1a4099-40df-45b1-8eda-fb25817123ba\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") "
Sep 17 18:27:33.007927 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.007913    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls\") pod \"5c1a4099-40df-45b1-8eda-fb25817123ba\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") "
Sep 17 18:27:33.008103 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.007954    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config\") pod \"5c1a4099-40df-45b1-8eda-fb25817123ba\" (UID: \"5c1a4099-40df-45b1-8eda-fb25817123ba\") "
Sep 17 18:27:33.008327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.008307    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5c1a4099-40df-45b1-8eda-fb25817123ba" (UID: "5c1a4099-40df-45b1-8eda-fb25817123ba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:27:33.010051 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.010019    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5c1a4099-40df-45b1-8eda-fb25817123ba" (UID: "5c1a4099-40df-45b1-8eda-fb25817123ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:27:33.010152 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.010050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw" (OuterVolumeSpecName: "kube-api-access-8w5gw") pod "5c1a4099-40df-45b1-8eda-fb25817123ba" (UID: "5c1a4099-40df-45b1-8eda-fb25817123ba"). InnerVolumeSpecName "kube-api-access-8w5gw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:27:33.108734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.108694    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:27:33.108734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.108727    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w5gw\" (UniqueName: \"kubernetes.io/projected/5c1a4099-40df-45b1-8eda-fb25817123ba-kube-api-access-8w5gw\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:27:33.108734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.108737    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5c1a4099-40df-45b1-8eda-fb25817123ba-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:27:33.274560 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.274467    2566 generic.go:356] "Generic (PLEG): container finished" podID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerID="7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1" exitCode=0
Sep 17 18:27:33.274560 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.274514    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"
Sep 17 18:27:33.275077 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.274556    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerDied","Data":"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"}
Sep 17 18:27:33.275077 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.274621    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx" event={"ID":"5c1a4099-40df-45b1-8eda-fb25817123ba","Type":"ContainerDied","Data":"3dc06eed6af4a095473f188a1db586d6e4a1a5b1667ed3b1a24606b96cc6a99a"}
Sep 17 18:27:33.275077 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.274649    2566 scope.go:117] "RemoveContainer" containerID="6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"
Sep 17 18:27:33.275077 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.275045    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:33.276685 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.276660    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:27:33.283021 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.283006    2566 scope.go:117] "RemoveContainer" containerID="7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"
Sep 17 18:27:33.289897 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.289881    2566 scope.go:117] "RemoveContainer" containerID="6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"
Sep 17 18:27:33.290167 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:27:33.290152    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92\": container with ID starting with 6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92 not found: ID does not exist" containerID="6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"
Sep 17 18:27:33.290204 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.290174    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92"} err="failed to get container status \"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92\": rpc error: code = NotFound desc = could not find container \"6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92\": container with ID starting with 6b76d98ff014d052ad34b4ecafcc314443b82dd7eea854da69c7d284f885cb92 not found: ID does not exist"
Sep 17 18:27:33.290204 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.290190    2566 scope.go:117] "RemoveContainer" containerID="7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"
Sep 17 18:27:33.290439 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:27:33.290414    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1\": container with ID starting with 7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1 not found: ID does not exist" containerID="7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"
Sep 17 18:27:33.290481 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.290443    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1"} err="failed to get container status \"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1\": rpc error: code = NotFound desc = could not find container \"7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1\": container with ID starting with 7b7f426c5b7a75478a3747b73c5978b4290dad7aaa5f8e4aa7f57acdb2b6f6d1 not found: ID does not exist"
Sep 17 18:27:33.296416 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.296392    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:27:33.297857 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.297838    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-2aa9e-predictor-777dd588ff-bgjgx"]
Sep 17 18:27:33.653515 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:33.653477    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" path="/var/lib/kubelet/pods/5c1a4099-40df-45b1-8eda-fb25817123ba/volumes"
Sep 17 18:27:34.279504 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:34.279470    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:27:37.512411 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:37.512320    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:39.284197 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:39.284163    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:27:39.284705 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:39.284681    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:27:42.512029 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:42.511991    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:42.512481 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:42.512115    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:27:47.511898 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:47.511857    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:49.285008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:49.284969    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:27:52.511797 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:52.511752    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:57.511971 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:57.511933    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:27:59.285173 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:59.285137    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:27:59.893309 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:27:59.893284    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:28:00.000521 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.000490    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls\") pod \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") "
Sep 17 18:28:00.000670 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.000542    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle\") pod \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\" (UID: \"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0\") "
Sep 17 18:28:00.000951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.000929    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" (UID: "a6220a9c-5fa3-499b-aa5b-46243ff5b9a0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:28:00.002516 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.002495    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" (UID: "a6220a9c-5fa3-499b-aa5b-46243ff5b9a0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:28:00.101193 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.101163    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:00.101193 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.101190    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:00.374746 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.374657    2566 generic.go:356] "Generic (PLEG): container finished" podID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerID="94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0" exitCode=0
Sep 17 18:28:00.374746 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.374701    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"
Sep 17 18:28:00.374746 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.374736    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" event={"ID":"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0","Type":"ContainerDied","Data":"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"}
Sep 17 18:28:00.375240 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.374799    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc" event={"ID":"a6220a9c-5fa3-499b-aa5b-46243ff5b9a0","Type":"ContainerDied","Data":"df0bda330b949b657cc0e15475014616ead23c726249dc0f6d060680e47a1d2b"}
Sep 17 18:28:00.375240 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.374822    2566 scope.go:117] "RemoveContainer" containerID="94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"
Sep 17 18:28:00.383153 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.383136    2566 scope.go:117] "RemoveContainer" containerID="94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"
Sep 17 18:28:00.383400 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:00.383383    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0\": container with ID starting with 94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0 not found: ID does not exist" containerID="94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"
Sep 17 18:28:00.383445 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.383408    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0"} err="failed to get container status \"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0\": rpc error: code = NotFound desc = could not find container \"94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0\": container with ID starting with 94acefffcdd12a3d776a2526944c0b1cadb80aa3d92069beab5bb4d71b5064d0 not found: ID does not exist"
Sep 17 18:28:00.395953 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.395930    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:28:00.399829 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:00.399809    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-2aa9e-744c775558-zn5kc"]
Sep 17 18:28:01.655629 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:01.655600    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" path="/var/lib/kubelet/pods/a6220a9c-5fa3-499b-aa5b-46243ff5b9a0/volumes"
Sep 17 18:28:09.285354 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:09.285315    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.31:8080: connect: connection refused"
Sep 17 18:28:19.285958 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:19.285922    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:28:26.507832 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.507800    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:28:26.508204 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.508008    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" containerID="cri-o://4c37cf7da9be5ae2dfd4f41eefa6a5bb77929fc6ea06f29c2320977679e9df67" gracePeriod=30
Sep 17 18:28:26.655181 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.655148    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:28:26.655421 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.655379    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" containerID="cri-o://b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9" gracePeriod=30
Sep 17 18:28:26.655497 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.655411    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kube-rbac-proxy" containerID="cri-o://c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed" gracePeriod=30
Sep 17 18:28:26.773031 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773005    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:28:26.773297 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773285    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container"
Sep 17 18:28:26.773338 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773298    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container"
Sep 17 18:28:26.773338 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773311    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e"
Sep 17 18:28:26.773338 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773317    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e"
Sep 17 18:28:26.773338 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773324    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kube-rbac-proxy"
Sep 17 18:28:26.773338 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773330    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kube-rbac-proxy"
Sep 17 18:28:26.773533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773375    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kserve-container"
Sep 17 18:28:26.773533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773383    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6220a9c-5fa3-499b-aa5b-46243ff5b9a0" containerName="ensemble-graph-2aa9e"
Sep 17 18:28:26.773533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.773391    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c1a4099-40df-45b1-8eda-fb25817123ba" containerName="kube-rbac-proxy"
Sep 17 18:28:26.776120 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.776104    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.778635 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.778614    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8365d-kube-rbac-proxy-sar-config\""
Sep 17 18:28:26.778718 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.778673    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8365d-predictor-serving-cert\""
Sep 17 18:28:26.782650 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.782620    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.782743 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.782708    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4ps\" (UniqueName: \"kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.782815 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.782749    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.785024 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.785005    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:28:26.883645 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.883612    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4ps\" (UniqueName: \"kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.883830 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.883655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.883830 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.883690    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.884423 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.884402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.886137 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.886116    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:26.899030 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:26.899010    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4ps\" (UniqueName: \"kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps\") pod \"success-200-isvc-8365d-predictor-69558d5f59-868nq\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") " pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:27.086623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.086591    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:27.358606 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.358529    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:28:27.361553 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:28:27.361528    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca99eb08_36a9_4220_a77b_1a870a8575de.slice/crio-1f8edf601bc0df1156afb408741095e2e686ce1df73c6bdd920e3623fbf97fab WatchSource:0}: Error finding container 1f8edf601bc0df1156afb408741095e2e686ce1df73c6bdd920e3623fbf97fab: Status 404 returned error can't find the container with id 1f8edf601bc0df1156afb408741095e2e686ce1df73c6bdd920e3623fbf97fab
Sep 17 18:28:27.472237 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.472208    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerStarted","Data":"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"}
Sep 17 18:28:27.472347 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.472267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerStarted","Data":"1f8edf601bc0df1156afb408741095e2e686ce1df73c6bdd920e3623fbf97fab"}
Sep 17 18:28:27.474191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.474169    2566 generic.go:356] "Generic (PLEG): container finished" podID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerID="c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed" exitCode=2
Sep 17 18:28:27.474302 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:27.474221    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerDied","Data":"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"}
Sep 17 18:28:28.479720 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:28.479684    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerStarted","Data":"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"}
Sep 17 18:28:28.480154 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:28.479765    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:28.497812 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:28.497743    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podStartSLOduration=2.497727188 podStartE2EDuration="2.497727188s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:28:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:28:28.496704068 +0000 UTC m=+1926.919887962" watchObservedRunningTime="2026-09-17 18:28:28.497727188 +0000 UTC m=+1926.920911082"
Sep 17 18:28:28.721118 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:28.721081    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:29.482907 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.482873    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:29.484237 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.484204    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:28:29.492067 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.492046    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.28:8643/healthz\": dial tcp 10.132.0.28:8643: connect: connection refused"
Sep 17 18:28:29.496532 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.496512    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.28:8080: connect: connection refused"
Sep 17 18:28:29.980469 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.980436    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:28:29.983595 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.983579    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:29.986292 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.986265    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-e0a3a-serving-cert\""
Sep 17 18:28:29.986401 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.986345    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-e0a3a-kube-rbac-proxy-sar-config\""
Sep 17 18:28:29.994038 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:29.994017    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:28:30.006168 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.006145    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.006257 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.006179    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.106760 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.106719    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.106945 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.106773    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.106945 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:30.106919    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-e0a3a-serving-cert: secret "splitter-graph-e0a3a-serving-cert" not found
Sep 17 18:28:30.107080 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:30.106999    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls podName:c0ec414f-cecb-4c77-85e7-78ef03bbc608 nodeName:}" failed. No retries permitted until 2026-09-17 18:28:30.606976925 +0000 UTC m=+1929.030160798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls") pod "splitter-graph-e0a3a-6b8c749bb4-pbx25" (UID: "c0ec414f-cecb-4c77-85e7-78ef03bbc608") : secret "splitter-graph-e0a3a-serving-cert" not found
Sep 17 18:28:30.107411 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.107387    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.486031 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.485990    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:28:30.611309 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.611272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.613629 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.613610    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") pod \"splitter-graph-e0a3a-6b8c749bb4-pbx25\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") " pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:30.894243 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:30.894207    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:31.164596 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:31.164569    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:28:31.167088 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:28:31.167061    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ec414f_cecb_4c77_85e7_78ef03bbc608.slice/crio-4c25ba0964863638a86586af339ab91471cee7e05ca61da5a9a90c5c62e6011c WatchSource:0}: Error finding container 4c25ba0964863638a86586af339ab91471cee7e05ca61da5a9a90c5c62e6011c: Status 404 returned error can't find the container with id 4c25ba0964863638a86586af339ab91471cee7e05ca61da5a9a90c5c62e6011c
Sep 17 18:28:31.491034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:31.490952    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" event={"ID":"c0ec414f-cecb-4c77-85e7-78ef03bbc608","Type":"ContainerStarted","Data":"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"}
Sep 17 18:28:31.491034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:31.491003    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" event={"ID":"c0ec414f-cecb-4c77-85e7-78ef03bbc608","Type":"ContainerStarted","Data":"4c25ba0964863638a86586af339ab91471cee7e05ca61da5a9a90c5c62e6011c"}
Sep 17 18:28:31.491405 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:31.491043    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:31.507941 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:31.507896    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podStartSLOduration=2.50788033 podStartE2EDuration="2.50788033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:28:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:28:31.5063202 +0000 UTC m=+1929.929504106" watchObservedRunningTime="2026-09-17 18:28:31.50788033 +0000 UTC m=+1929.931064221"
Sep 17 18:28:33.100957 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.100932    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:28:33.129229 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.129201    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls\") pod \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") "
Sep 17 18:28:33.129367 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.129240    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzxvj\" (UniqueName: \"kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj\") pod \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") "
Sep 17 18:28:33.129367 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.129319    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config\") pod \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\" (UID: \"7d45d1ae-0e56-4f48-98d9-79211d1757eb\") "
Sep 17 18:28:33.129715 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.129693    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7d45d1ae-0e56-4f48-98d9-79211d1757eb" (UID: "7d45d1ae-0e56-4f48-98d9-79211d1757eb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:28:33.131316 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.131292    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7d45d1ae-0e56-4f48-98d9-79211d1757eb" (UID: "7d45d1ae-0e56-4f48-98d9-79211d1757eb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:28:33.131401 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.131293    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj" (OuterVolumeSpecName: "kube-api-access-tzxvj") pod "7d45d1ae-0e56-4f48-98d9-79211d1757eb" (UID: "7d45d1ae-0e56-4f48-98d9-79211d1757eb"). InnerVolumeSpecName "kube-api-access-tzxvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:28:33.230161 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.230079    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:33.230161 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.230105    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7d45d1ae-0e56-4f48-98d9-79211d1757eb-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:33.230161 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.230114    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzxvj\" (UniqueName: \"kubernetes.io/projected/7d45d1ae-0e56-4f48-98d9-79211d1757eb-kube-api-access-tzxvj\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:33.499677 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.499591    2566 generic.go:356] "Generic (PLEG): container finished" podID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerID="b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9" exitCode=0
Sep 17 18:28:33.499677 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.499642    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"
Sep 17 18:28:33.499892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.499678    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerDied","Data":"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"}
Sep 17 18:28:33.499892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.499735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf" event={"ID":"7d45d1ae-0e56-4f48-98d9-79211d1757eb","Type":"ContainerDied","Data":"0433fcd6372a1369b69e8a74d9d84d834f54dc93ddafc63c32adef1dc3b5f0d8"}
Sep 17 18:28:33.499892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.499763    2566 scope.go:117] "RemoveContainer" containerID="c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"
Sep 17 18:28:33.507994 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.507975    2566 scope.go:117] "RemoveContainer" containerID="b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"
Sep 17 18:28:33.515096 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.515076    2566 scope.go:117] "RemoveContainer" containerID="c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"
Sep 17 18:28:33.515356 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:33.515337    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed\": container with ID starting with c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed not found: ID does not exist" containerID="c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"
Sep 17 18:28:33.515424 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.515368    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed"} err="failed to get container status \"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed\": rpc error: code = NotFound desc = could not find container \"c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed\": container with ID starting with c6f0650f109b291d60c834cf5d033b7fa5ea1af7ae7602d1a736bc4a361d10ed not found: ID does not exist"
Sep 17 18:28:33.515424 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.515393    2566 scope.go:117] "RemoveContainer" containerID="b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"
Sep 17 18:28:33.515665 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:33.515648    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9\": container with ID starting with b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9 not found: ID does not exist" containerID="b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"
Sep 17 18:28:33.515704 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.515672    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9"} err="failed to get container status \"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9\": rpc error: code = NotFound desc = could not find container \"b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9\": container with ID starting with b2137b0df63f43d3f69864e7552b7980c7323ff0cd606c00ca723bd0d15f04a9 not found: ID does not exist"
Sep 17 18:28:33.520245 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.520223    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:28:33.524865 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.524821    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-04b70-predictor-697dcfdbb9-k58nf"]
Sep 17 18:28:33.653620 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.653582    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" path="/var/lib/kubelet/pods/7d45d1ae-0e56-4f48-98d9-79211d1757eb/volumes"
Sep 17 18:28:33.721373 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:33.721339    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:35.491154 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:35.491125    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:28:35.491801 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:35.491756    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:28:37.500255 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:37.500228    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:38.720884 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:38.720847    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:38.721313 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:38.720969    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:28:40.068378 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.068341    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:28:40.068856 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.068565    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" containerID="cri-o://724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979" gracePeriod=30
Sep 17 18:28:40.151026 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.150986    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:28:40.151219 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.151198    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container" containerID="cri-o://865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7" gracePeriod=30
Sep 17 18:28:40.151401 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.151259    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kube-rbac-proxy" containerID="cri-o://0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8" gracePeriod=30
Sep 17 18:28:40.261308 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261257    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:28:40.261732 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261714    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kube-rbac-proxy"
Sep 17 18:28:40.261732 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261735    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kube-rbac-proxy"
Sep 17 18:28:40.261877 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261770    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container"
Sep 17 18:28:40.261877 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261793    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container"
Sep 17 18:28:40.261877 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261867    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kube-rbac-proxy"
Sep 17 18:28:40.262054 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.261881    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d45d1ae-0e56-4f48-98d9-79211d1757eb" containerName="kserve-container"
Sep 17 18:28:40.267333 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.267300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.270409 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.270374    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1a268-predictor-serving-cert\""
Sep 17 18:28:40.270540 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.270387    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-1a268-kube-rbac-proxy-sar-config\""
Sep 17 18:28:40.276682 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.276638    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:28:40.390276 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.390079    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.390276 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.390160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5gr\" (UniqueName: \"kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.390276 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.390197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.490684 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.490629    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.490951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.490698    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5gr\" (UniqueName: \"kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.490951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.490738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.490951 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:40.490794    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-1a268-predictor-serving-cert: secret "success-200-isvc-1a268-predictor-serving-cert" not found
Sep 17 18:28:40.490951 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:40.490894    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls podName:915c78f7-5b12-4673-863f-24a81bf6ef7b nodeName:}" failed. No retries permitted until 2026-09-17 18:28:40.990869529 +0000 UTC m=+1939.414053416 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls") pod "success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" (UID: "915c78f7-5b12-4673-863f-24a81bf6ef7b") : secret "success-200-isvc-1a268-predictor-serving-cert" not found
Sep 17 18:28:40.491453 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.491434    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.501399 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.501371    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5gr\" (UniqueName: \"kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.527125 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.527097    2566 generic.go:356] "Generic (PLEG): container finished" podID="bd07cc66-7379-42e7-972f-11236706b28d" containerID="0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8" exitCode=2
Sep 17 18:28:40.527258 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.527162    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerDied","Data":"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"}
Sep 17 18:28:40.995190 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.995140    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:40.997578 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:40.997544    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") pod \"success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") " pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:41.182653 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:41.182617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:41.452881 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:41.452856    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:28:41.455585 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:28:41.455557    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod915c78f7_5b12_4673_863f_24a81bf6ef7b.slice/crio-1f5ac27864008fa575d8fbcc37d88705bfda9bdc66d284a298267786cc465ff3 WatchSource:0}: Error finding container 1f5ac27864008fa575d8fbcc37d88705bfda9bdc66d284a298267786cc465ff3: Status 404 returned error can't find the container with id 1f5ac27864008fa575d8fbcc37d88705bfda9bdc66d284a298267786cc465ff3
Sep 17 18:28:41.533445 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:41.533419    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerStarted","Data":"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"}
Sep 17 18:28:41.533533 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:41.533477    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerStarted","Data":"1f5ac27864008fa575d8fbcc37d88705bfda9bdc66d284a298267786cc465ff3"}
Sep 17 18:28:42.498893 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.498852    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:42.538405 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.538373    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerStarted","Data":"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"}
Sep 17 18:28:42.538648 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.538634    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:42.538812 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.538795    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:42.540124 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.540099    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:28:42.557398 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:42.557349    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podStartSLOduration=2.557334339 podStartE2EDuration="2.557334339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:28:42.555484971 +0000 UTC m=+1940.978668863" watchObservedRunningTime="2026-09-17 18:28:42.557334339 +0000 UTC m=+1940.980518233"
Sep 17 18:28:43.493480 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.493456    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:28:43.543520 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.543486    2566 generic.go:356] "Generic (PLEG): container finished" podID="bd07cc66-7379-42e7-972f-11236706b28d" containerID="865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7" exitCode=0
Sep 17 18:28:43.543976 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.543549    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"
Sep 17 18:28:43.543976 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.543563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerDied","Data":"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"}
Sep 17 18:28:43.543976 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.543645    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg" event={"ID":"bd07cc66-7379-42e7-972f-11236706b28d","Type":"ContainerDied","Data":"45d64cee60c7598bdb358749254172ffc5989d28b05173d44159c73d6eead443"}
Sep 17 18:28:43.543976 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.543676    2566 scope.go:117] "RemoveContainer" containerID="0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"
Sep 17 18:28:43.544189 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.544102    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:28:43.553448 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.553430    2566 scope.go:117] "RemoveContainer" containerID="865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"
Sep 17 18:28:43.560993 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.560968    2566 scope.go:117] "RemoveContainer" containerID="0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"
Sep 17 18:28:43.561284 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:43.561265    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8\": container with ID starting with 0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8 not found: ID does not exist" containerID="0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"
Sep 17 18:28:43.561355 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.561298    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8"} err="failed to get container status \"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8\": rpc error: code = NotFound desc = could not find container \"0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8\": container with ID starting with 0a836e5c7b4ebab75c8b55099a57d3a8af0b561bd7e711e97b67812d882426f8 not found: ID does not exist"
Sep 17 18:28:43.561355 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.561323    2566 scope.go:117] "RemoveContainer" containerID="865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"
Sep 17 18:28:43.561610 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:28:43.561591    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7\": container with ID starting with 865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7 not found: ID does not exist" containerID="865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"
Sep 17 18:28:43.561652 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.561618    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7"} err="failed to get container status \"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7\": rpc error: code = NotFound desc = could not find container \"865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7\": container with ID starting with 865178dd9e73973b34e4bf16f9d7c9af79099db20dec110251f463af35c417f7 not found: ID does not exist"
Sep 17 18:28:43.616315 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.616219    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") pod \"bd07cc66-7379-42e7-972f-11236706b28d\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") "
Sep 17 18:28:43.616315 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.616279    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config\") pod \"bd07cc66-7379-42e7-972f-11236706b28d\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") "
Sep 17 18:28:43.616315 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.616316    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cjzv\" (UniqueName: \"kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv\") pod \"bd07cc66-7379-42e7-972f-11236706b28d\" (UID: \"bd07cc66-7379-42e7-972f-11236706b28d\") "
Sep 17 18:28:43.616698 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.616670    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bd07cc66-7379-42e7-972f-11236706b28d" (UID: "bd07cc66-7379-42e7-972f-11236706b28d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:28:43.618417 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.618387    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv" (OuterVolumeSpecName: "kube-api-access-5cjzv") pod "bd07cc66-7379-42e7-972f-11236706b28d" (UID: "bd07cc66-7379-42e7-972f-11236706b28d"). InnerVolumeSpecName "kube-api-access-5cjzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:28:43.618417 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.618410    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bd07cc66-7379-42e7-972f-11236706b28d" (UID: "bd07cc66-7379-42e7-972f-11236706b28d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:28:43.717313 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.717282    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bd07cc66-7379-42e7-972f-11236706b28d-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:43.717313 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.717312    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bd07cc66-7379-42e7-972f-11236706b28d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:43.717521 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.717397    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cjzv\" (UniqueName: \"kubernetes.io/projected/bd07cc66-7379-42e7-972f-11236706b28d-kube-api-access-5cjzv\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:43.720757 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.720726    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:43.860326 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.860290    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:28:43.865212 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:43.865182    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-e0a3a-predictor-6f4c696847-pr6tg"]
Sep 17 18:28:44.549376 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:44.549334    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:28:45.491725 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:45.491685    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:28:45.654274 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:45.654242    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd07cc66-7379-42e7-972f-11236706b28d" path="/var/lib/kubelet/pods/bd07cc66-7379-42e7-972f-11236706b28d/volumes"
Sep 17 18:28:47.499430 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:47.499391    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:48.721334 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:48.721297    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:49.554476 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:49.554440    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:28:49.554942 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:49.554916    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:28:52.499010 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:52.498966    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:52.499381 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:52.499067    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:28:53.721071 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:53.721036    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:55.492136 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:55.492095    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:28:56.598191 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.598161    2566 generic.go:356] "Generic (PLEG): container finished" podID="3b0d4577-7276-4745-a654-989995e8d7ac" containerID="4c37cf7da9be5ae2dfd4f41eefa6a5bb77929fc6ea06f29c2320977679e9df67" exitCode=0
Sep 17 18:28:56.598518 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.598229    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" event={"ID":"3b0d4577-7276-4745-a654-989995e8d7ac","Type":"ContainerDied","Data":"4c37cf7da9be5ae2dfd4f41eefa6a5bb77929fc6ea06f29c2320977679e9df67"}
Sep 17 18:28:56.645665 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.645637    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:28:56.721445 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.721407    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle\") pod \"3b0d4577-7276-4745-a654-989995e8d7ac\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") "
Sep 17 18:28:56.721604 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.721499    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls\") pod \"3b0d4577-7276-4745-a654-989995e8d7ac\" (UID: \"3b0d4577-7276-4745-a654-989995e8d7ac\") "
Sep 17 18:28:56.721748 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.721725    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3b0d4577-7276-4745-a654-989995e8d7ac" (UID: "3b0d4577-7276-4745-a654-989995e8d7ac"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:28:56.723472 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.723444    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3b0d4577-7276-4745-a654-989995e8d7ac" (UID: "3b0d4577-7276-4745-a654-989995e8d7ac"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:28:56.822287 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.822263    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3b0d4577-7276-4745-a654-989995e8d7ac-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:56.822287 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:56.822286    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b0d4577-7276-4745-a654-989995e8d7ac-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:28:57.499051 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.499012    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:28:57.603477 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.603450    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"
Sep 17 18:28:57.603477 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.603470    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv" event={"ID":"3b0d4577-7276-4745-a654-989995e8d7ac","Type":"ContainerDied","Data":"79a16311ac102ca355e70db90216542493ff99d8a8729819e36f7eb07401f0dd"}
Sep 17 18:28:57.603940 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.603519    2566 scope.go:117] "RemoveContainer" containerID="4c37cf7da9be5ae2dfd4f41eefa6a5bb77929fc6ea06f29c2320977679e9df67"
Sep 17 18:28:57.623616 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.623588    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:28:57.627708 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.627687    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-04b70-6564cd7dcf-j6jhv"]
Sep 17 18:28:57.652623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:57.652594    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" path="/var/lib/kubelet/pods/3b0d4577-7276-4745-a654-989995e8d7ac/volumes"
Sep 17 18:28:59.555842 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:28:59.555809    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:29:02.498891 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:02.498852    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:29:05.492524 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:05.492486    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.32:8080: connect: connection refused"
Sep 17 18:29:07.498975 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:07.498897    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:29:09.555733 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:09.555696    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:29:10.216843 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.216815    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:29:10.317017 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.316982    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") pod \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") "
Sep 17 18:29:10.317181 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.317028    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle\") pod \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\" (UID: \"c0ec414f-cecb-4c77-85e7-78ef03bbc608\") "
Sep 17 18:29:10.317406 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.317384    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c0ec414f-cecb-4c77-85e7-78ef03bbc608" (UID: "c0ec414f-cecb-4c77-85e7-78ef03bbc608"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:29:10.319145 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.319123    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c0ec414f-cecb-4c77-85e7-78ef03bbc608" (UID: "c0ec414f-cecb-4c77-85e7-78ef03bbc608"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:29:10.418346 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.418314    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0ec414f-cecb-4c77-85e7-78ef03bbc608-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:29:10.418346 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.418340    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c0ec414f-cecb-4c77-85e7-78ef03bbc608-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:29:10.654379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.654290    2566 generic.go:356] "Generic (PLEG): container finished" podID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerID="724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979" exitCode=0
Sep 17 18:29:10.654379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.654321    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" event={"ID":"c0ec414f-cecb-4c77-85e7-78ef03bbc608","Type":"ContainerDied","Data":"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"}
Sep 17 18:29:10.654379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.654340    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"
Sep 17 18:29:10.654379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.654375    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25" event={"ID":"c0ec414f-cecb-4c77-85e7-78ef03bbc608","Type":"ContainerDied","Data":"4c25ba0964863638a86586af339ab91471cee7e05ca61da5a9a90c5c62e6011c"}
Sep 17 18:29:10.655107 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.654386    2566 scope.go:117] "RemoveContainer" containerID="724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"
Sep 17 18:29:10.662478 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.662459    2566 scope.go:117] "RemoveContainer" containerID="724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"
Sep 17 18:29:10.662729 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:29:10.662715    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979\": container with ID starting with 724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979 not found: ID does not exist" containerID="724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"
Sep 17 18:29:10.662775 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.662737    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979"} err="failed to get container status \"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979\": rpc error: code = NotFound desc = could not find container \"724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979\": container with ID starting with 724b713418aa21dd1171684626af528768f06d23b2205da3de72accfde5cc979 not found: ID does not exist"
Sep 17 18:29:10.674794 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.674758    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:29:10.680500 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:10.680479    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-e0a3a-6b8c749bb4-pbx25"]
Sep 17 18:29:11.654090 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:11.654056    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" path="/var/lib/kubelet/pods/c0ec414f-cecb-4c77-85e7-78ef03bbc608/volumes"
Sep 17 18:29:15.492892 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:15.492860    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:29:19.555227 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:19.555188    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.34:8080: connect: connection refused"
Sep 17 18:29:26.762950 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.762919    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763172    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kube-rbac-proxy"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763184    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kube-rbac-proxy"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763196    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763202    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763217    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763223    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763229    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763236    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763276    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kube-rbac-proxy"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763283    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0ec414f-cecb-4c77-85e7-78ef03bbc608" containerName="splitter-graph-e0a3a"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763291    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b0d4577-7276-4745-a654-989995e8d7ac" containerName="sequence-graph-04b70"
Sep 17 18:29:26.763327 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.763297    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd07cc66-7379-42e7-972f-11236706b28d" containerName="kserve-container"
Sep 17 18:29:26.767371 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.767352    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:26.769811 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.769775    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8365d-kube-rbac-proxy-sar-config\""
Sep 17 18:29:26.770028 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.770016    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8365d-serving-cert\""
Sep 17 18:29:26.774118 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.774099    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:29:26.932107 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.932066    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:26.932107 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:26.932109    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.032756 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.032670    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.032756 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.032711    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.032941 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:29:27.032852    2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-8365d-serving-cert: secret "switch-graph-8365d-serving-cert" not found
Sep 17 18:29:27.032941 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:29:27.032906    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls podName:bdc3c2ea-88ae-4489-b0a0-5546072639e4 nodeName:}" failed. No retries permitted until 2026-09-17 18:29:27.532889754 +0000 UTC m=+1985.956073627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls") pod "switch-graph-8365d-79d674c554-6js2m" (UID: "bdc3c2ea-88ae-4489-b0a0-5546072639e4") : secret "switch-graph-8365d-serving-cert" not found
Sep 17 18:29:27.033369 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.033348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.537667 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.537618    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.540155 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.540129    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") pod \"switch-graph-8365d-79d674c554-6js2m\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") " pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.677999 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.677969    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:27.945996 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:27.945815    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:29:27.948494 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:29:27.948464    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc3c2ea_88ae_4489_b0a0_5546072639e4.slice/crio-a69906e2922063121555510263cab7c6d9a2efed5528aff743c5c55e3b2ce6b1 WatchSource:0}: Error finding container a69906e2922063121555510263cab7c6d9a2efed5528aff743c5c55e3b2ce6b1: Status 404 returned error can't find the container with id a69906e2922063121555510263cab7c6d9a2efed5528aff743c5c55e3b2ce6b1
Sep 17 18:29:28.717658 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:28.717614    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" event={"ID":"bdc3c2ea-88ae-4489-b0a0-5546072639e4","Type":"ContainerStarted","Data":"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"}
Sep 17 18:29:28.717850 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:28.717675    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" event={"ID":"bdc3c2ea-88ae-4489-b0a0-5546072639e4","Type":"ContainerStarted","Data":"a69906e2922063121555510263cab7c6d9a2efed5528aff743c5c55e3b2ce6b1"}
Sep 17 18:29:28.717850 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:28.717706    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:28.734536 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:28.734493    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podStartSLOduration=2.73447712 podStartE2EDuration="2.73447712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:29:28.733584413 +0000 UTC m=+1987.156768322" watchObservedRunningTime="2026-09-17 18:29:28.73447712 +0000 UTC m=+1987.157661012"
Sep 17 18:29:29.555626 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:29.555598    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:29:34.726228 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:34.726201    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:29:40.295312 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.295278    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:29:40.298353 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.298333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.301255 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.301238    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-1a268-kube-rbac-proxy-sar-config\""
Sep 17 18:29:40.301339 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.301268    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-1a268-serving-cert\""
Sep 17 18:29:40.306527 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.306505    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:29:40.338963 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.338939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.339082 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.338976    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.439591 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.439554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.439739 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.439599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.440169 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.440149    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.441985 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.441969    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls\") pod \"splitter-graph-1a268-7579f4dc47-xgz4s\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") " pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.609023 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.608915    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:40.879017 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:40.878992    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:29:40.881544 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:29:40.881512    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98dbd592_220b_4edc_b16b_b6018853225e.slice/crio-64af033724fc8d32784c80bbe54fbd63d30e1ecc5a55657c31feb95c1488500c WatchSource:0}: Error finding container 64af033724fc8d32784c80bbe54fbd63d30e1ecc5a55657c31feb95c1488500c: Status 404 returned error can't find the container with id 64af033724fc8d32784c80bbe54fbd63d30e1ecc5a55657c31feb95c1488500c
Sep 17 18:29:41.767738 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:41.767703    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" event={"ID":"98dbd592-220b-4edc-b16b-b6018853225e","Type":"ContainerStarted","Data":"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"}
Sep 17 18:29:41.768113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:41.767759    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" event={"ID":"98dbd592-220b-4edc-b16b-b6018853225e","Type":"ContainerStarted","Data":"64af033724fc8d32784c80bbe54fbd63d30e1ecc5a55657c31feb95c1488500c"}
Sep 17 18:29:41.768113 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:41.767891    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:29:41.783688 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:41.783594    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podStartSLOduration=1.783577254 podStartE2EDuration="1.783577254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:29:41.783251263 +0000 UTC m=+2000.206435158" watchObservedRunningTime="2026-09-17 18:29:41.783577254 +0000 UTC m=+2000.206761151"
Sep 17 18:29:47.776606 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:29:47.776577    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:37:54.739585 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:54.739553    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:37:54.742036 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:54.739737    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" containerID="cri-o://8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e" gracePeriod=30
Sep 17 18:37:54.858457 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:54.858426    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:37:54.858680 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:54.858654    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container" containerID="cri-o://b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9" gracePeriod=30
Sep 17 18:37:54.858759 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:54.858710    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kube-rbac-proxy" containerID="cri-o://c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb" gracePeriod=30
Sep 17 18:37:55.509996 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:55.509961    2566 generic.go:356] "Generic (PLEG): container finished" podID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerID="c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb" exitCode=2
Sep 17 18:37:55.510164 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:55.510015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerDied","Data":"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"}
Sep 17 18:37:57.775468 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:57.775424    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:37:58.098073 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.098050    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:37:58.145873 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.145838    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") pod \"915c78f7-5b12-4673-863f-24a81bf6ef7b\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") "
Sep 17 18:37:58.146046 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.145878    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config\") pod \"915c78f7-5b12-4673-863f-24a81bf6ef7b\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") "
Sep 17 18:37:58.146046 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.145943    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2h5gr\" (UniqueName: \"kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr\") pod \"915c78f7-5b12-4673-863f-24a81bf6ef7b\" (UID: \"915c78f7-5b12-4673-863f-24a81bf6ef7b\") "
Sep 17 18:37:58.146283 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.146260    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "915c78f7-5b12-4673-863f-24a81bf6ef7b" (UID: "915c78f7-5b12-4673-863f-24a81bf6ef7b"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:37:58.148018 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.147989    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "915c78f7-5b12-4673-863f-24a81bf6ef7b" (UID: "915c78f7-5b12-4673-863f-24a81bf6ef7b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:37:58.148121 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.148035    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr" (OuterVolumeSpecName: "kube-api-access-2h5gr") pod "915c78f7-5b12-4673-863f-24a81bf6ef7b" (UID: "915c78f7-5b12-4673-863f-24a81bf6ef7b"). InnerVolumeSpecName "kube-api-access-2h5gr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:37:58.246734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.246701    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2h5gr\" (UniqueName: \"kubernetes.io/projected/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-api-access-2h5gr\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:37:58.246734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.246728    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/915c78f7-5b12-4673-863f-24a81bf6ef7b-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:37:58.246734 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.246738    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/915c78f7-5b12-4673-863f-24a81bf6ef7b-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:37:58.526198 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.526106    2566 generic.go:356] "Generic (PLEG): container finished" podID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerID="b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9" exitCode=0
Sep 17 18:37:58.526198 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.526165    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"
Sep 17 18:37:58.526390 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.526190    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerDied","Data":"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"}
Sep 17 18:37:58.526390 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.526272    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s" event={"ID":"915c78f7-5b12-4673-863f-24a81bf6ef7b","Type":"ContainerDied","Data":"1f5ac27864008fa575d8fbcc37d88705bfda9bdc66d284a298267786cc465ff3"}
Sep 17 18:37:58.526390 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.526300    2566 scope.go:117] "RemoveContainer" containerID="c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"
Sep 17 18:37:58.534887 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.534866    2566 scope.go:117] "RemoveContainer" containerID="b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"
Sep 17 18:37:58.541710 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.541693    2566 scope.go:117] "RemoveContainer" containerID="c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"
Sep 17 18:37:58.541973 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:37:58.541951    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb\": container with ID starting with c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb not found: ID does not exist" containerID="c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"
Sep 17 18:37:58.542034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.541980    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb"} err="failed to get container status \"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb\": rpc error: code = NotFound desc = could not find container \"c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb\": container with ID starting with c54de5c3248fe89e41271740cb38e95f56c72428d6b5327f9dd954cbfd57fbbb not found: ID does not exist"
Sep 17 18:37:58.542034 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.541997    2566 scope.go:117] "RemoveContainer" containerID="b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"
Sep 17 18:37:58.542229 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:37:58.542211    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9\": container with ID starting with b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9 not found: ID does not exist" containerID="b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"
Sep 17 18:37:58.542270 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.542237    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9"} err="failed to get container status \"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9\": rpc error: code = NotFound desc = could not find container \"b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9\": container with ID starting with b421e7ef8cd631488409711917b00a63b675c5052f9a596ed76c08de1a26c9f9 not found: ID does not exist"
Sep 17 18:37:58.547501 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.547476    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:37:58.552876 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:58.552855    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-1a268-predictor-7c75dbcd8c-pk25s"]
Sep 17 18:37:59.651758 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:37:59.651715    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" path="/var/lib/kubelet/pods/915c78f7-5b12-4673-863f-24a81bf6ef7b/volumes"
Sep 17 18:38:02.775367 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:02.775331    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:38:07.775287 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:07.775204    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:38:07.775660 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:07.775307    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:38:12.775673 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:12.775629    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:38:17.775180 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:17.775140    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:38:22.775289 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:22.775243    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:38:25.381724 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.381701    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:38:25.438047 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.438014    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls\") pod \"98dbd592-220b-4edc-b16b-b6018853225e\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") "
Sep 17 18:38:25.438223 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.438086    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle\") pod \"98dbd592-220b-4edc-b16b-b6018853225e\" (UID: \"98dbd592-220b-4edc-b16b-b6018853225e\") "
Sep 17 18:38:25.438457 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.438431    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "98dbd592-220b-4edc-b16b-b6018853225e" (UID: "98dbd592-220b-4edc-b16b-b6018853225e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:38:25.440141 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.440120    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "98dbd592-220b-4edc-b16b-b6018853225e" (UID: "98dbd592-220b-4edc-b16b-b6018853225e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:38:25.539449 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.539419    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/98dbd592-220b-4edc-b16b-b6018853225e-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:38:25.539449 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.539448    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98dbd592-220b-4edc-b16b-b6018853225e-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:38:25.627754 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.627724    2566 generic.go:356] "Generic (PLEG): container finished" podID="98dbd592-220b-4edc-b16b-b6018853225e" containerID="8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e" exitCode=0
Sep 17 18:38:25.627948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.627768    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"
Sep 17 18:38:25.627948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.627801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" event={"ID":"98dbd592-220b-4edc-b16b-b6018853225e","Type":"ContainerDied","Data":"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"}
Sep 17 18:38:25.627948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.627855    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s" event={"ID":"98dbd592-220b-4edc-b16b-b6018853225e","Type":"ContainerDied","Data":"64af033724fc8d32784c80bbe54fbd63d30e1ecc5a55657c31feb95c1488500c"}
Sep 17 18:38:25.627948 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.627885    2566 scope.go:117] "RemoveContainer" containerID="8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"
Sep 17 18:38:25.636157 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.636137    2566 scope.go:117] "RemoveContainer" containerID="8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"
Sep 17 18:38:25.636432 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:38:25.636411    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e\": container with ID starting with 8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e not found: ID does not exist" containerID="8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"
Sep 17 18:38:25.636512 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.636440    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e"} err="failed to get container status \"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e\": rpc error: code = NotFound desc = could not find container \"8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e\": container with ID starting with 8fe53fa2cff047777ce299f815282cf521184ebb9f4699c39c82dea9191d162e not found: ID does not exist"
Sep 17 18:38:25.656007 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.655979    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:38:25.659749 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:25.659727    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-1a268-7579f4dc47-xgz4s"]
Sep 17 18:38:27.652061 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:38:27.652023    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98dbd592-220b-4edc-b16b-b6018853225e" path="/var/lib/kubelet/pods/98dbd592-220b-4edc-b16b-b6018853225e/volumes"
Sep 17 18:45:45.817311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:45.817236    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:45:45.819752 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:45.817420    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" containerID="cri-o://f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7" gracePeriod=30
Sep 17 18:45:45.937467 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:45.937430    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:45:45.937661 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:45.937636    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container" containerID="cri-o://374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a" gracePeriod=30
Sep 17 18:45:45.937733 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:45.937712    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kube-rbac-proxy" containerID="cri-o://0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822" gracePeriod=30
Sep 17 18:45:46.138139 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:46.138047    2566 generic.go:356] "Generic (PLEG): container finished" podID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerID="0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822" exitCode=2
Sep 17 18:45:46.138139 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:46.138122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerDied","Data":"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"}
Sep 17 18:45:48.974479 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:48.974455    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:45:49.071946 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.071909    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq4ps\" (UniqueName: \"kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps\") pod \"ca99eb08-36a9-4220-a77b-1a870a8575de\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") "
Sep 17 18:45:49.072130 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.071955    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls\") pod \"ca99eb08-36a9-4220-a77b-1a870a8575de\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") "
Sep 17 18:45:49.072130 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.072007    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config\") pod \"ca99eb08-36a9-4220-a77b-1a870a8575de\" (UID: \"ca99eb08-36a9-4220-a77b-1a870a8575de\") "
Sep 17 18:45:49.072403 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.072374    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ca99eb08-36a9-4220-a77b-1a870a8575de" (UID: "ca99eb08-36a9-4220-a77b-1a870a8575de"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:45:49.074082 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.074050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps" (OuterVolumeSpecName: "kube-api-access-wq4ps") pod "ca99eb08-36a9-4220-a77b-1a870a8575de" (UID: "ca99eb08-36a9-4220-a77b-1a870a8575de"). InnerVolumeSpecName "kube-api-access-wq4ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 18:45:49.074162 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.074055    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ca99eb08-36a9-4220-a77b-1a870a8575de" (UID: "ca99eb08-36a9-4220-a77b-1a870a8575de"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:45:49.150014 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.149982    2566 generic.go:356] "Generic (PLEG): container finished" podID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerID="374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a" exitCode=0
Sep 17 18:45:49.150178 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.150049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerDied","Data":"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"}
Sep 17 18:45:49.150178 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.150115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq" event={"ID":"ca99eb08-36a9-4220-a77b-1a870a8575de","Type":"ContainerDied","Data":"1f8edf601bc0df1156afb408741095e2e686ce1df73c6bdd920e3623fbf97fab"}
Sep 17 18:45:49.150178 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.150051    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"
Sep 17 18:45:49.150178 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.150140    2566 scope.go:117] "RemoveContainer" containerID="0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"
Sep 17 18:45:49.160591 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.160568    2566 scope.go:117] "RemoveContainer" containerID="374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"
Sep 17 18:45:49.167767 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.167749    2566 scope.go:117] "RemoveContainer" containerID="0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"
Sep 17 18:45:49.168047 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:45:49.168031    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822\": container with ID starting with 0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822 not found: ID does not exist" containerID="0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"
Sep 17 18:45:49.168094 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.168056    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822"} err="failed to get container status \"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822\": rpc error: code = NotFound desc = could not find container \"0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822\": container with ID starting with 0d7dbc69a141f13c4d150e5d6152a14863b8cb8f8cd455811e3d040523e8f822 not found: ID does not exist"
Sep 17 18:45:49.168094 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.168072    2566 scope.go:117] "RemoveContainer" containerID="374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"
Sep 17 18:45:49.168303 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:45:49.168289    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a\": container with ID starting with 374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a not found: ID does not exist" containerID="374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"
Sep 17 18:45:49.168340 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.168307    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a"} err="failed to get container status \"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a\": rpc error: code = NotFound desc = could not find container \"374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a\": container with ID starting with 374d5c295229428bd556f1ec57285af721bd4d70cde59f3fa59fa37d5e47d96a not found: ID does not exist"
Sep 17 18:45:49.171973 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.171951    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:45:49.172937 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.172919    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4ps\" (UniqueName: \"kubernetes.io/projected/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-api-access-wq4ps\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:45:49.172989 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.172939    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ca99eb08-36a9-4220-a77b-1a870a8575de-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:45:49.172989 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.172949    2566 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ca99eb08-36a9-4220-a77b-1a870a8575de-kube-rbac-proxy-sar-config\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:45:49.178161 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.178142    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8365d-predictor-69558d5f59-868nq"]
Sep 17 18:45:49.651701 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.651670    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" path="/var/lib/kubelet/pods/ca99eb08-36a9-4220-a77b-1a870a8575de/volumes"
Sep 17 18:45:49.724587 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:49.724547    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:45:54.725369 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:54.725336    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:45:59.724757 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:59.724713    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:45:59.725176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:45:59.724844    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:46:03.544982 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:03.544952    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:04.369371 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:04.369342    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:04.725498 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:04.725412    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:46:05.191135 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:05.191099    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:05.990485 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:05.990455    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:06.788012 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:06.787976    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:07.622384 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:07.622348    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:08.437379 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:08.437346    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:09.261357 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:09.261323    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:09.724594 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:09.724556    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:46:10.049996 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:10.049967    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:10.844177 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:10.844145    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:11.640341 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:11.640306    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:12.443800 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:12.443755    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:13.249142 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:13.249108    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:14.062932 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:14.062903    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:14.724938 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:14.724904    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 17 18:46:14.942067 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:14.942029    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-8365d-79d674c554-6js2m_bdc3c2ea-88ae-4489-b0a0-5546072639e4/switch-graph-8365d/0.log"
Sep 17 18:46:15.953271 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:15.953246    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:46:16.066082 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.066046    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle\") pod \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") "
Sep 17 18:46:16.066082 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.066089    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") pod \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\" (UID: \"bdc3c2ea-88ae-4489-b0a0-5546072639e4\") "
Sep 17 18:46:16.066541 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.066514    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bdc3c2ea-88ae-4489-b0a0-5546072639e4" (UID: "bdc3c2ea-88ae-4489-b0a0-5546072639e4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 18:46:16.068332 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.068310    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bdc3c2ea-88ae-4489-b0a0-5546072639e4" (UID: "bdc3c2ea-88ae-4489-b0a0-5546072639e4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 18:46:16.166855 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.166817    2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdc3c2ea-88ae-4489-b0a0-5546072639e4-openshift-service-ca-bundle\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:46:16.166855 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.166846    2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdc3c2ea-88ae-4489-b0a0-5546072639e4-proxy-tls\") on node \"ip-10-0-135-193.ec2.internal\" DevicePath \"\""
Sep 17 18:46:16.237911 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.237876    2566 generic.go:356] "Generic (PLEG): container finished" podID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerID="f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7" exitCode=0
Sep 17 18:46:16.238072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.237922    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"
Sep 17 18:46:16.238072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.237932    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" event={"ID":"bdc3c2ea-88ae-4489-b0a0-5546072639e4","Type":"ContainerDied","Data":"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"}
Sep 17 18:46:16.238072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.237973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m" event={"ID":"bdc3c2ea-88ae-4489-b0a0-5546072639e4","Type":"ContainerDied","Data":"a69906e2922063121555510263cab7c6d9a2efed5528aff743c5c55e3b2ce6b1"}
Sep 17 18:46:16.238072 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.237993    2566 scope.go:117] "RemoveContainer" containerID="f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"
Sep 17 18:46:16.245658 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.245583    2566 scope.go:117] "RemoveContainer" containerID="f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"
Sep 17 18:46:16.246028 ip-10-0-135-193 kubenswrapper[2566]: E0917 18:46:16.246003    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7\": container with ID starting with f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7 not found: ID does not exist" containerID="f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"
Sep 17 18:46:16.246142 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.246034    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7"} err="failed to get container status \"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7\": rpc error: code = NotFound desc = could not find container \"f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7\": container with ID starting with f83332c72858614211505e45fd98166ee3f6afe23fdb4904c0ef3cf5420448d7 not found: ID does not exist"
Sep 17 18:46:16.258876 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.258850    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:46:16.262033 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:16.262010    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8365d-79d674c554-6js2m"]
Sep 17 18:46:17.652345 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:17.652305    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" path="/var/lib/kubelet/pods/bdc3c2ea-88ae-4489-b0a0-5546072639e4/volumes"
Sep 17 18:46:18.839921 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.839888    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bs2q6/must-gather-7klxq"]
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840115    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840125    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840133    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840139    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840149    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840154    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840163    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840168    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840180    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840186    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840195    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840200    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840248    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840255    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kube-rbac-proxy"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840262    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc3c2ea-88ae-4489-b0a0-5546072639e4" containerName="switch-graph-8365d"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840267    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="98dbd592-220b-4edc-b16b-b6018853225e" containerName="splitter-graph-1a268"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840274    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca99eb08-36a9-4220-a77b-1a870a8575de" containerName="kserve-container"
Sep 17 18:46:18.840311 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.840279    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="915c78f7-5b12-4673-863f-24a81bf6ef7b" containerName="kserve-container"
Sep 17 18:46:18.844135 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.844120    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:18.846877 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.846858    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bs2q6\"/\"kube-root-ca.crt\""
Sep 17 18:46:18.846951 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.846860    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bs2q6\"/\"openshift-service-ca.crt\""
Sep 17 18:46:18.848078 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.848063    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bs2q6\"/\"default-dockercfg-5rqxp\""
Sep 17 18:46:18.851454 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.851433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs2q6/must-gather-7klxq"]
Sep 17 18:46:18.984868 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.984828    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bcca4958-282f-4c7f-8a28-4b85357036e3-must-gather-output\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:18.985040 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:18.984886    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nltq\" (UniqueName: \"kubernetes.io/projected/bcca4958-282f-4c7f-8a28-4b85357036e3-kube-api-access-4nltq\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.085994 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.085959    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nltq\" (UniqueName: \"kubernetes.io/projected/bcca4958-282f-4c7f-8a28-4b85357036e3-kube-api-access-4nltq\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.086156 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.086010    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bcca4958-282f-4c7f-8a28-4b85357036e3-must-gather-output\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.086290 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.086276    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/bcca4958-282f-4c7f-8a28-4b85357036e3-must-gather-output\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.094323 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.094261    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nltq\" (UniqueName: \"kubernetes.io/projected/bcca4958-282f-4c7f-8a28-4b85357036e3-kube-api-access-4nltq\") pod \"must-gather-7klxq\" (UID: \"bcca4958-282f-4c7f-8a28-4b85357036e3\") " pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.153376 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.153341    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs2q6/must-gather-7klxq"
Sep 17 18:46:19.420349 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.420270    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs2q6/must-gather-7klxq"]
Sep 17 18:46:19.423464 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:46:19.423439    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcca4958_282f_4c7f_8a28_4b85357036e3.slice/crio-e11977ee2cee55bd69791798f265af137031383d43643975b3c0a9da7743d143 WatchSource:0}: Error finding container e11977ee2cee55bd69791798f265af137031383d43643975b3c0a9da7743d143: Status 404 returned error can't find the container with id e11977ee2cee55bd69791798f265af137031383d43643975b3c0a9da7743d143
Sep 17 18:46:19.425317 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:19.425301    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 18:46:20.256725 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:20.256699    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs2q6/must-gather-7klxq" event={"ID":"bcca4958-282f-4c7f-8a28-4b85357036e3","Type":"ContainerStarted","Data":"e11977ee2cee55bd69791798f265af137031383d43643975b3c0a9da7743d143"}
Sep 17 18:46:21.262912 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.262872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs2q6/must-gather-7klxq" event={"ID":"bcca4958-282f-4c7f-8a28-4b85357036e3","Type":"ContainerStarted","Data":"b3ea70c198839cfa8fc9783dd8b0bf5ae84ba3181626fe418da75157aca864bf"}
Sep 17 18:46:21.262912 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.262926    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs2q6/must-gather-7klxq" event={"ID":"bcca4958-282f-4c7f-8a28-4b85357036e3","Type":"ContainerStarted","Data":"28084b868a37af6245a8a682ecd117e77d353a0e32025d5281ba2e736697613a"}
Sep 17 18:46:21.279025 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.278971    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bs2q6/must-gather-7klxq" podStartSLOduration=2.534762428 podStartE2EDuration="3.278950593s" totalImagesPullingTime="744.188165ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 18:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:46:21.2784842 +0000 UTC m=+2999.701668096" watchObservedRunningTime="2026-09-17 18:46:21.278950593 +0000 UTC m=+2999.702134491"
Sep 17 18:46:21.613865 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.613808    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qfd6w_0b23d48e-e957-4278-b558-443a13430040/global-pull-secret-syncer/0.log"
Sep 17 18:46:21.774348 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.774319    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wg4sr_8dfb0243-cd9b-4843-b14a-4bde217501c8/konnectivity-agent/0.log"
Sep 17 18:46:21.816601 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:21.816572    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-193.ec2.internal_46e5674716c4f3612e5b47f16efafbea/haproxy/0.log"
Sep 17 18:46:26.367071 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:26.367035    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96427_057bcc10-81c2-4d75-90a8-3ea8a5c59c30/node-exporter/0.log"
Sep 17 18:46:26.396143 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:26.396117    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96427_057bcc10-81c2-4d75-90a8-3ea8a5c59c30/kube-rbac-proxy/0.log"
Sep 17 18:46:26.425383 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:26.425344    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-96427_057bcc10-81c2-4d75-90a8-3ea8a5c59c30/init-textfile/0.log"
Sep 17 18:46:29.660576 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.660540    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"]
Sep 17 18:46:29.664727 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.664699    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.673054 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.673025    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"]
Sep 17 18:46:29.781932 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.781896    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv57d\" (UniqueName: \"kubernetes.io/projected/0c2b0857-c585-472d-addb-75e7f84c01ee-kube-api-access-zv57d\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.782226 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.782206    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-lib-modules\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.782453 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.782424    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-podres\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.782638 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.782622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-proc\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.782995 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.782970    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-sys\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.883987 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.883932    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-proc\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.883998    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-sys\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884075    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-proc\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv57d\" (UniqueName: \"kubernetes.io/projected/0c2b0857-c585-472d-addb-75e7f84c01ee-kube-api-access-zv57d\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884153    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-sys\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884176 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884153    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-lib-modules\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884455 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884213    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-podres\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884455 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884312    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-lib-modules\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.884455 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.884339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0c2b0857-c585-472d-addb-75e7f84c01ee-podres\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.893674 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.893637    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv57d\" (UniqueName: \"kubernetes.io/projected/0c2b0857-c585-472d-addb-75e7f84c01ee-kube-api-access-zv57d\") pod \"perf-node-gather-daemonset-rv4sx\" (UID: \"0c2b0857-c585-472d-addb-75e7f84c01ee\") " pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:29.978996 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:29.978920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:30.256230 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:30.255207    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"]
Sep 17 18:46:30.258545 ip-10-0-135-193 kubenswrapper[2566]: W0917 18:46:30.258505    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0c2b0857_c585_472d_addb_75e7f84c01ee.slice/crio-1ec088e77f796307fe5300b1d76c638e16552dc0cbfc11507ceb309d7496a35a WatchSource:0}: Error finding container 1ec088e77f796307fe5300b1d76c638e16552dc0cbfc11507ceb309d7496a35a: Status 404 returned error can't find the container with id 1ec088e77f796307fe5300b1d76c638e16552dc0cbfc11507ceb309d7496a35a
Sep 17 18:46:30.306404 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:30.306368    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx" event={"ID":"0c2b0857-c585-472d-addb-75e7f84c01ee","Type":"ContainerStarted","Data":"1ec088e77f796307fe5300b1d76c638e16552dc0cbfc11507ceb309d7496a35a"}
Sep 17 18:46:30.746250 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:30.746216    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6ttvb_82f4c1a3-81b6-4005-b4f7-271643238f04/dns/0.log"
Sep 17 18:46:30.767086 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:30.767056    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6ttvb_82f4c1a3-81b6-4005-b4f7-271643238f04/kube-rbac-proxy/0.log"
Sep 17 18:46:30.922236 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:30.922203    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7s5jk_a1515c65-4fb6-4f0a-9e62-649e22abdbff/dns-node-resolver/0.log"
Sep 17 18:46:31.312498 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:31.312462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx" event={"ID":"0c2b0857-c585-472d-addb-75e7f84c01ee","Type":"ContainerStarted","Data":"c959597eac70138a0b1e12edaa9cf9f151704496437c751640d7a60723f33045"}
Sep 17 18:46:31.312705 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:31.312603    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:31.331529 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:31.331476    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx" podStartSLOduration=2.331459853 podStartE2EDuration="2.331459853s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 18:46:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 18:46:31.330275846 +0000 UTC m=+3009.753459749" watchObservedRunningTime="2026-09-17 18:46:31.331459853 +0000 UTC m=+3009.754643747"
Sep 17 18:46:31.485978 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:31.485951    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-prs4c_e9062e02-a63c-4406-9782-5c0095128a9e/node-ca/0.log"
Sep 17 18:46:32.560244 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:32.560195    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qrq4m_470a672f-2cd1-4459-ad70-0d7b5b96624f/serve-healthcheck-canary/0.log"
Sep 17 18:46:32.998121 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:32.998022    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqm4_6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82/kube-rbac-proxy/0.log"
Sep 17 18:46:33.017623 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:33.017596    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqm4_6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82/exporter/0.log"
Sep 17 18:46:33.038515 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:33.038457    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8mqm4_6a7bc5eb-2132-4c1e-80c8-5ab8f5d78d82/extractor/0.log"
Sep 17 18:46:35.388468 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:35.388439    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-9hl52_79d7922c-3d2d-420e-b331-79336c240a64/server/0.log"
Sep 17 18:46:35.557158 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:35.557104    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-cxw8b_ef490b6b-72b2-4827-801d-da8172b303cf/seaweedfs/0.log"
Sep 17 18:46:37.325265 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:37.325230    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bs2q6/perf-node-gather-daemonset-rv4sx"
Sep 17 18:46:41.436080 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.436014    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/kube-multus-additional-cni-plugins/0.log"
Sep 17 18:46:41.457949 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.457916    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/egress-router-binary-copy/0.log"
Sep 17 18:46:41.476427 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.476400    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/cni-plugins/0.log"
Sep 17 18:46:41.495586 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.495559    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/bond-cni-plugin/0.log"
Sep 17 18:46:41.514370 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.514340    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/routeoverride-cni/0.log"
Sep 17 18:46:41.534871 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.534845    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/whereabouts-cni-bincopy/0.log"
Sep 17 18:46:41.552714 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.552687    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rtjw7_9cad02b4-c802-4273-b90c-55f9ca072e76/whereabouts-cni/0.log"
Sep 17 18:46:41.594069 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.594039    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bwbk5_cd2d6bef-a10f-42e0-b02e-b73f3510a19b/kube-multus/0.log"
Sep 17 18:46:41.665209 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.665172    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8bqps_97e76c9b-8f82-49a0-9714-fded16b71ee7/network-metrics-daemon/0.log"
Sep 17 18:46:41.682063 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:41.682040    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8bqps_97e76c9b-8f82-49a0-9714-fded16b71ee7/kube-rbac-proxy/0.log"
Sep 17 18:46:42.682369 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.682324    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/ovn-controller/0.log"
Sep 17 18:46:42.696527 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.696502    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/ovn-acl-logging/0.log"
Sep 17 18:46:42.720008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.719986    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/kube-rbac-proxy-node/0.log"
Sep 17 18:46:42.738008 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.737984    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 18:46:42.758022 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.757996    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/northd/0.log"
Sep 17 18:46:42.777422 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.777380    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/nbdb/0.log"
Sep 17 18:46:42.796917 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.796890    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/sbdb/0.log"
Sep 17 18:46:42.854448 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:42.854420    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dspj8_8989de48-e3c3-40e1-b0cb-1db2ea9b570d/ovnkube-controller/0.log"
Sep 17 18:46:44.158453 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:44.158424    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-6zqk5_bc3acc2b-d024-4416-936c-1d752ad0e43e/network-check-target-container/0.log"
Sep 17 18:46:45.144691 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:45.144661    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qdf8r_855b4e5c-fbda-4f66-accd-a5dbac52c006/iptables-alerter/0.log"
Sep 17 18:46:45.869644 ip-10-0-135-193 kubenswrapper[2566]: I0917 18:46:45.869616    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-c7zh2_5c0b01b6-4b66-4b41-873e-e8be72a209e8/tuned/0.log"