Sep 16 17:22:36.069371 ip-10-0-134-96 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 17:22:36.069384 ip-10-0-134-96 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 17:22:36.069394 ip-10-0-134-96 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 17:22:36.070663 ip-10-0-134-96 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 17:22:46.259689 ip-10-0-134-96 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 17:22:46.259706 ip-10-0-134-96 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 1ba7e27d54f144f3b849f4a27c9d1a65 --
Sep 16 17:24:19.903124 ip-10-0-134-96 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 17:24:20.313296 ip-10-0-134-96 kubenswrapper[2542]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:20.313296 ip-10-0-134-96 kubenswrapper[2542]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 17:24:20.313296 ip-10-0-134-96 kubenswrapper[2542]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:20.313296 ip-10-0-134-96 kubenswrapper[2542]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 17:24:20.313296 ip-10-0-134-96 kubenswrapper[2542]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 17:24:20.316063 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.315982    2542 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 17:24:20.320121 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320106    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.320121 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320121    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320125    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320129    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320132    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320135    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320139    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320142    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320144    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320147    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320150    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320154    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320157    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320160    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320162    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320166    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320169    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320171    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320174    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320177    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320180    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.320186 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320182    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320185    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320188    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320190    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320193    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320196    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320199    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320202    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320205    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320208    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320211    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320214    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320216    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320219    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320221    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320224    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320229    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320233    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320236    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.320745 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320239    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320242    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320244    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320246    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320251    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320254    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320257    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320260    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320263    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320265    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320267    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320270    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320273    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320275    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320279    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320282    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320285    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320287    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320290    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320293    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.321232 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320295    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320298    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320300    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320303    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320307    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320310    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320312    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320315    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320317    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320320    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320322    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320325    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320327    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320330    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320332    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320335    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320337    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320340    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320342    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320345    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.321709 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320348    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320350    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320353    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320356    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320358    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320361    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320364    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320367    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320370    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320373    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320375    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320379    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320383    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320386    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320389    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320392    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320395    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320398    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.320400    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.322249 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321126    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321133    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321135    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321138    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321141    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321143    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321146    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321150    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321154    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321157    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321160    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321163    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321166    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321170    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321173    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321175    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321178    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321182    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.322703 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321185    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321189    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321192    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321194    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321197    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321199    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321202    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321205    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321207    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321209    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321212    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321215    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321217    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321220    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321223    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321225    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321228    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321230    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321233    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321236    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.323277 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321238    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321241    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321243    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321246    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321248    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321251    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321254    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321256    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321259    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321262    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321265    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321267    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321269    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321272    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321275    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321278    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321280    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321283    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321285    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321288    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.323799 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321290    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321292    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321295    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321297    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321301    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321304    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321306    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321309    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321311    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321313    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321316    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321321    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321324    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321326    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321329    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321331    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321334    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321337    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321339    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321342    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.324317 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321344    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321349    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321352    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321354    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321357    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321359    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321362    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321365    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321367    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321370    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321372    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321376    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321379    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321382    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321384    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321387    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321390    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321392    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321395    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321398    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.324816 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.321400    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323305    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323320    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323324    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323327    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323330    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323333    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323336    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323339    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323341    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323344    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323347    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323349    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323352    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323354    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323358    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323363    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323366    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323368    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323371    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.325289 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323374    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323376    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323380    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323383    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323385    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323388    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323390    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323393    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323395    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323398    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323401    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323403    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323406    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323409    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323414    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323416    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323419    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323421    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323424    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323426    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.325772 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323428    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323431    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323434    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323436    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323438    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323441    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323443    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323446    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323448    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323451    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323453    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323456    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323458    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323461    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323463    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323466    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323468    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323470    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323473    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323475    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.326260 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323478    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323480    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323483    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323485    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323488    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323492    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323496    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323499    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323503    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323507    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323510    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323513    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323515    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323518    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323521    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323524    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323526    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323528    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323531    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.326725 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323533    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323542    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323545    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323548    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323550    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323562    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323565    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323567    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323570    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323591    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323594    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323596    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323599    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323601    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323604    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323607    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323609    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323612    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323618    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.327184 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323621    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.323624    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323697    2542 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323743    2542 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323751    2542 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323755    2542 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323760    2542 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323763    2542 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323767    2542 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323771    2542 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323775    2542 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323778    2542 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323781    2542 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323784    2542 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323787    2542 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323790    2542 flags.go:64] FLAG: --cgroup-root=""
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323793    2542 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323796    2542 flags.go:64] FLAG: --client-ca-file=""
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323799    2542 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323809    2542 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323813    2542 flags.go:64] FLAG: --cluster-domain=""
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323816    2542 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323819    2542 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323823    2542 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 17:24:20.327670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323826    2542 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323830    2542 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323833    2542 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323836    2542 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323839    2542 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323842    2542 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323845    2542 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323862    2542 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323865    2542 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323868    2542 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323873    2542 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323875    2542 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323878    2542 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323881    2542 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323884    2542 flags.go:64] FLAG: --enable-server="true"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323887    2542 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323894    2542 flags.go:64] FLAG: --event-burst="100"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323897    2542 flags.go:64] FLAG: --event-qps="50"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323900    2542 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323903    2542 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323906    2542 flags.go:64] FLAG: --eviction-hard=""
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323910    2542 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323912    2542 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323915    2542 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323918    2542 flags.go:64] FLAG: --eviction-soft=""
Sep 16 17:24:20.328311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323921    2542 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323924    2542 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323926    2542 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323935    2542 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323938    2542 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323941    2542 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323944    2542 flags.go:64] FLAG: --feature-gates=""
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323948    2542 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323951    2542 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323954    2542 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323957    2542 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323960    2542 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323963    2542 flags.go:64] FLAG: --help="false"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323966    2542 flags.go:64] FLAG: --hostname-override="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323971    2542 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323974    2542 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323977    2542 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323980    2542 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323984    2542 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323987    2542 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323990    2542 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323993    2542 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323996    2542 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 17:24:20.328936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.323999    2542 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324002    2542 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324005    2542 flags.go:64] FLAG: --kube-reserved=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324008    2542 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324011    2542 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324014    2542 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324016    2542 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324019    2542 flags.go:64] FLAG: --lock-file=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324022    2542 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324025    2542 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324028    2542 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324033    2542 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324036    2542 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324040    2542 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324043    2542 flags.go:64] FLAG: --logging-format="text"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324045    2542 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324049    2542 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324052    2542 flags.go:64] FLAG: --manifest-url=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324055    2542 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324059    2542 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324062    2542 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324066    2542 flags.go:64] FLAG: --max-pods="110"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324069    2542 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324074    2542 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324077    2542 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 17:24:20.329504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324079    2542 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324082    2542 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324085    2542 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324088    2542 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324096    2542 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324099    2542 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324102    2542 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324105    2542 flags.go:64] FLAG: --pod-cidr=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324108    2542 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324113    2542 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324116    2542 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324119    2542 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324122    2542 flags.go:64] FLAG: --port="10250"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324125    2542 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324128    2542 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03332a7da08f884c9"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324131    2542 flags.go:64] FLAG: --qos-reserved=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324134    2542 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324137    2542 flags.go:64] FLAG: --register-node="true"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324140    2542 flags.go:64] FLAG: --register-with-taints=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324143    2542 flags.go:64] FLAG: --registry-burst="10"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324146    2542 flags.go:64] FLAG: --registry-qps="5"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324149    2542 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324152    2542 flags.go:64] FLAG: --reserved-memory=""
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324156    2542 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324159    2542 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 17:24:20.330114 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324161    2542 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324164    2542 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324167    2542 flags.go:64] FLAG: --runonce="false"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324170    2542 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324173    2542 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324177    2542 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324181    2542 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324184    2542 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324187    2542 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324190    2542 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324193    2542 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324196    2542 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324199    2542 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324202    2542 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324205    2542 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324208    2542 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324211    2542 flags.go:64] FLAG: --system-cgroups=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324214    2542 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324219    2542 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324222    2542 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324225    2542 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324231    2542 flags.go:64] FLAG: --tls-min-version=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324234    2542 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324237    2542 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324240    2542 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 17:24:20.330706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324243    2542 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324246    2542 flags.go:64] FLAG: --v="2"
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324250    2542 flags.go:64] FLAG: --version="false"
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324254    2542 flags.go:64] FLAG: --vmodule=""
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324258    2542 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.324261    2542 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324369    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324373    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324376    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324379    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324382    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324385    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324390    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324393    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324397    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324399    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324402    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324405    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324408    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324410    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324413    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.331419 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324416    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324418    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324421    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324424    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324426    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324429    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324432    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324434    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324437    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324439    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324442    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324444    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324447    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324450    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324453    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324455    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324458    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324461    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324463    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324466    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.332015 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324469    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324472    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324474    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324478    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324482    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324484    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324487    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324490    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324494    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324497    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324501    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324505    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324508    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324511    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324514    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324516    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324519    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324523    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324526    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.332501 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324529    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324532    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324535    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324537    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324540    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324542    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324545    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324548    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324551    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324553    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324557    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324559    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324562    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324565    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324567    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324570    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324574    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324578    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324581    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324584    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.332987 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324586    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324589    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324592    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324595    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324597    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324600    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324602    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324605    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324608    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324610    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324613    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324616    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324618    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324621    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324623    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324626    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324629    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324631    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324634    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324637    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.333470 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324639    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324642    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324644    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324657    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.324660    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.325589    2542 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333125    2542 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333138    2542 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333159    2542 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333164    2542 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333225    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333230    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333233    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333236    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.333965 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333239    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333241    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333245    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333249    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333251    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333254    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333257    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333259    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333261    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333264    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333266    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333269    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333272    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333275    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333277    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333280    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333282    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333285    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333287    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.334303 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333290    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333292    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333295    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333298    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333301    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333303    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333307    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333310    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333313    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333315    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333318    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333320    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333322    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333325    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333327    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333331    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333335    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333339    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333341    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333344    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.334789 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333347    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333349    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333352    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333354    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333357    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333360    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333362    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333365    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333367    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333370    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333372    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333375    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333377    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333380    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333382    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333385    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333388    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333390    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333393    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.335346 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333401    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333404    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333406    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333410    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333414    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333417    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333419    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333423    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333425    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333428    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333431    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333433    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333436    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333439    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333441    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333443    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333446    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333448    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333451    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.335823 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333454    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333456    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333458    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333461    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333464    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333466    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333469    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333471    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333474    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333477    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333480    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333483    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333485    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333488    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333496    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333499    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333501    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.336285 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333504    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333509    2542 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333629    2542 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333634    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333638    2542 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333641    2542 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333643    2542 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333646    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333649    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333651    2542 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333654    2542 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333656    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333658    2542 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333661    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333663    2542 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 17:24:20.336699 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333665    2542 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333668    2542 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333670    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333674    2542 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333678    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333681    2542 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333684    2542 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333686    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333689    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333691    2542 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333695    2542 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333697    2542 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333700    2542 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333702    2542 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333705    2542 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333712    2542 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333714    2542 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333717    2542 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333720    2542 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333722    2542 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 17:24:20.337087 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333725    2542 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333727    2542 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333730    2542 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333732    2542 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333735    2542 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333737    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333739    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333742    2542 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333745    2542 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333747    2542 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333750    2542 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333752    2542 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333755    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333757    2542 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333760    2542 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333762    2542 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333764    2542 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333767    2542 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333769    2542 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 17:24:20.337552 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333772    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333774    2542 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333777    2542 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333779    2542 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333782    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333785    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333787    2542 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333791    2542 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333794    2542 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333797    2542 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333800    2542 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333803    2542 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333806    2542 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333808    2542 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333811    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333813    2542 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333816    2542 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333818    2542 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333821    2542 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 17:24:20.338020 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333823    2542 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333825    2542 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333828    2542 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333831    2542 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333833    2542 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333836    2542 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333838    2542 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333841    2542 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333843    2542 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333846    2542 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333865    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333869    2542 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333873    2542 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333876    2542 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333879    2542 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333881    2542 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333884    2542 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333886    2542 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333889    2542 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 17:24:20.338485 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333893    2542 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333896    2542 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333899    2542 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333902    2542 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333905    2542 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333908    2542 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333911    2542 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333914    2542 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:20.333917    2542 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.333922    2542 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 17:24:20.338976 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.334015    2542 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 17:24:20.339241 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.339004    2542 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 17:24:20.339991 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.339979    2542 server.go:1019] "Starting client certificate rotation"
Sep 16 17:24:20.340095 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.340079    2542 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 17:24:20.340130 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.340109    2542 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 17:24:20.342826 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.342813    2542 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 17:24:20.345802 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.345787    2542 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 17:24:20.358112 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.358088    2542 log.go:25] "Validated CRI v1 runtime API"
Sep 16 17:24:20.362031 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.362006    2542 log.go:25] "Validated CRI v1 image API"
Sep 16 17:24:20.364106 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.364089    2542 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 17:24:20.366601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.366578    2542 fs.go:135] Filesystem UUIDs: map[065d9a49-360f-461d-905a-c7ed9fbd14c0:/dev/nvme0n1p3 6382285e-5405-49ba-9cba-d42f35a3baaf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 16 17:24:20.366683 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.366600    2542 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 17:24:20.371697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.371563    2542 manager.go:216] Machine: {Timestamp:2026-09-16 17:24:20.37014199 +0000 UTC m=+0.362237575 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3201713 MemoryCapacity:33164464128 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23958ebd07bec58f770d223c3b6ede SystemUUID:ec23958e-bd07-bec5-8f77-0d223c3b6ede BootID:1ba7e27d-54f1-44f3-b849-f4a27c9d1a65 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582230016 Type:vfs Inodes:4048396 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:dd:3a:a1:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:dd:3a:a1:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:72:13:17:7d:e8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164464128 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 17:24:20.371697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.371684    2542 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 17:24:20.371875 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.371785    2542 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 17:24:20.372266 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.372249    2542 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 17:24:20.373930 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.373909    2542 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 17:24:20.374062 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.373934    2542 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-96.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 17:24:20.374891 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.374881    2542 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 17:24:20.374925 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.374893    2542 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 17:24:20.374925 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.374901    2542 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 17:24:20.375751 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.375741    2542 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 17:24:20.377009 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.377000    2542 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 17:24:20.381657 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381646    2542 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 17:24:20.381781 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381772    2542 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 17:24:20.381815 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381800    2542 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 17:24:20.381815 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381809    2542 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 17:24:20.381877 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381822    2542 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 17:24:20.381877 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381830    2542 kubelet.go:398] "Adding apiserver pod source"
Sep 16 17:24:20.381877 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.381839    2542 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 17:24:20.382188 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.382177    2542 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 17:24:20.382233 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.382194    2542 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 17:24:20.384468 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.384453    2542 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 17:24:20.387462 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.387449    2542 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 17:24:20.387524 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.387478    2542 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 17:24:20.389532 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389522    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389537    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389543    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389549    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389558    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389564    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 17:24:20.389578 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389577    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 17:24:20.389767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389583    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 17:24:20.389767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389589    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 17:24:20.389767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389604    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 17:24:20.389767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389620    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 17:24:20.389767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.389626    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 17:24:20.391233 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.391223    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 17:24:20.391268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.391234    2542 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 17:24:20.392346 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.392326    2542 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 17:24:20.392389 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.392363    2542 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 17:24:20.392637 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.392622    2542 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-96.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 17:24:20.395034 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.395024    2542 server.go:1268] "Started kubelet"
Sep 16 17:24:20.395159 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.395122    2542 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 17:24:20.395221 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.395101    2542 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 17:24:20.395221 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.395199    2542 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 17:24:20.395845 ip-10-0-134-96 systemd[1]: Started Kubernetes Kubelet.
Sep 16 17:24:20.398512 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.398489    2542 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 17:24:20.399266 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.399242    2542 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 17:24:20.400281 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.400211    2542 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 17:24:20.403686 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.402502    2542 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a22de0cb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.395000011 +0000 UTC m=+0.387095597,LastTimestamp:2026-09-16 17:24:20.395000011 +0000 UTC m=+0.387095597,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.405591 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.405572    2542 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 17:24:20.406308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.406287    2542 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 17:24:20.406453 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.406433    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.407339 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.407188    2542 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 17:24:20.407339 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.407190    2542 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 17:24:20.407428 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.407352    2542 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 17:24:20.407465 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.407444    2542 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 17:24:20.407465 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.407451    2542 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 17:24:20.409161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409149    2542 factory.go:153] Registering CRI-O factory
Sep 16 17:24:20.409161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409162    2542 factory.go:223] Registration of the crio container factory successfully
Sep 16 17:24:20.409248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409200    2542 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 17:24:20.409248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409206    2542 factory.go:55] Registering systemd factory
Sep 16 17:24:20.409248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409212    2542 factory.go:223] Registration of the systemd container factory successfully
Sep 16 17:24:20.409248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409226    2542 factory.go:103] Registering Raw factory
Sep 16 17:24:20.409248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409235    2542 manager.go:1196] Started watching for new ooms in manager
Sep 16 17:24:20.409551 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.409540    2542 manager.go:349] Starting recovery of all containers
Sep 16 17:24:20.418600 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.418571    2542 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-96.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 16 17:24:20.418737 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.418703    2542 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 16 17:24:20.421027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.421009    2542 manager.go:354] Recovery completed
Sep 16 17:24:20.425577 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.425565    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.427312 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.427294    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.427388 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.427327    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.427388 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.427341    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.428016 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428001    2542 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 17:24:20.428016 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428013    2542 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 17:24:20.428151 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428028    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.428151 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428031    2542 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 17:24:20.428723 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428707    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.428804 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.428736    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.429025 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.429008    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.442123 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.442046    2542 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41ae66f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427310703 +0000 UTC m=+0.419406292,LastTimestamp:2026-09-16 17:24:20.427310703 +0000 UTC m=+0.419406292,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.442925 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.442894    2542 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kzq2v"
Sep 16 17:24:20.450684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.450666    2542 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kzq2v"
Sep 16 17:24:20.464377 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.464302    2542 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41b3f52  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427333458 +0000 UTC m=+0.419429045,LastTimestamp:2026-09-16 17:24:20.427333458 +0000 UTC m=+0.419429045,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.476622    2542 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41b74c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427347137 +0000 UTC m=+0.419442723,LastTimestamp:2026-09-16 17:24:20.427347137 +0000 UTC m=+0.419442723,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.491261    2542 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-96.ec2.internal.18d5dd09a41ae66f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41ae66f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427310703 +0000 UTC m=+0.419406292,LastTimestamp:2026-09-16 17:24:20.428722357 +0000 UTC m=+0.420817949,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.501478    2542 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-96.ec2.internal.18d5dd09a41b3f52\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41b3f52  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427333458 +0000 UTC m=+0.419429045,LastTimestamp:2026-09-16 17:24:20.428742773 +0000 UTC m=+0.420838360,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.507231    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.511106    2542 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-96.ec2.internal.18d5dd09a41b74c1\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-96.ec2.internal.18d5dd09a41b74c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-96.ec2.internal,UID:ip-10-0-134-96.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-96.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-96.ec2.internal,},FirstTimestamp:2026-09-16 17:24:20.427347137 +0000 UTC m=+0.419442723,LastTimestamp:2026-09-16 17:24:20.429027393 +0000 UTC m=+0.421122989,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-96.ec2.internal,}"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.572869    2542 policy_none.go:49] "None policy: Start"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.572888    2542 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 17:24:20.573838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.572902    2542 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 17:24:20.575699 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.575686    2542 policy_none.go:47] "Start"
Sep 16 17:24:20.607295 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.607274    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.607381 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607365    2542 manager.go:337] "Starting Device Plugin manager"
Sep 16 17:24:20.607436 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.607402    2542 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 17:24:20.607436 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607416    2542 server.go:87] "Starting device plugin registration server"
Sep 16 17:24:20.607736 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607722    2542 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 17:24:20.607806 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607735    2542 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 17:24:20.607872 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607844    2542 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 17:24:20.607951 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607929    2542 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 17:24:20.607951 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.607944    2542 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 17:24:20.608567 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.608540    2542 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 17:24:20.608639 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.608590    2542 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.616284 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.616264    2542 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 17:24:20.617364 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.617344    2542 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 17:24:20.617433 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.617397    2542 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 17:24:20.617433 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.617422    2542 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 17:24:20.617497 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.617464    2542 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 17:24:20.624285 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.624266    2542 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:20.628532 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.628517    2542 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-96.ec2.internal\" not found" node="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.708121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.708103    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.708984 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.708971    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.709041 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.708997    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.709041 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.709007    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.709041 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.709028    2542 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.717652 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.717636    2542 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"]
Sep 16 17:24:20.717696 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.717690    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.717804 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.717780    2542 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.717839 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.717804    2542 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-96.ec2.internal\": node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.718472 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.718461    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.718519 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.718486    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.718519 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.718499    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.720915 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.720904    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.721044 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.721029    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.721089 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.721057    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.723051 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723035    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.723194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723064    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.723194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723076    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.723194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723124    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.723194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723142    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.723194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.723153    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.725438 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.725424    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.725504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.725448    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:20.726225 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.726211    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:20.726308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.726237    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:20.726308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.726251    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:20.733493 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.733479    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.750340 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.750324    2542 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-96.ec2.internal\" not found" node="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.754601 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.754587    2542 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-96.ec2.internal\" not found" node="ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.809607 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.809585    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.809607 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.809608    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.809738 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.809624    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95d3581c8d80d714714e1182983c2fc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-96.ec2.internal\" (UID: \"d95d3581c8d80d714714e1182983c2fc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.833911 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.833873    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:20.910329 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910310    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.910409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910333    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.910409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910348    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95d3581c8d80d714714e1182983c2fc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-96.ec2.internal\" (UID: \"d95d3581c8d80d714714e1182983c2fc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.910409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910395    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d95d3581c8d80d714714e1182983c2fc-config\") pod \"kube-apiserver-proxy-ip-10-0-134-96.ec2.internal\" (UID: \"d95d3581c8d80d714714e1182983c2fc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.910522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910422    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.910522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:20.910437    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e71972a41c7eb6801726c8cf0011774d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal\" (UID: \"e71972a41c7eb6801726c8cf0011774d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:20.934428 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:20.934411    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.035264 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.035235    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.053394 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.053376    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:21.057308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.057281    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:21.135502 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.135437    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.235933 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.235901    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.336715 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.336692    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.340130 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.340107    2542 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 17:24:21.340265 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.340245    2542 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 17:24:21.366573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.366550    2542 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:21.437340 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.437279    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.511088 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.511062    2542 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 17:24:21.512166 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.512150    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 17:24:21.512268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.512185    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 17:24:21.512268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.512201    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeHasSufficientPID"
Sep 16 17:24:21.538390 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.538365    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.576122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.576090    2542 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 17:19:20 +0000 UTC" deadline="2028-03-26 15:00:49.597770467 +0000 UTC"
Sep 16 17:24:21.576122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.576117    2542 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13365h36m28.021656631s"
Sep 16 17:24:21.639982 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:21.639954    2542 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-96.ec2.internal\" not found"
Sep 16 17:24:21.739774 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.739705    2542 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:21.793263 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:21.793217    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95d3581c8d80d714714e1182983c2fc.slice/crio-005df66637c9429378bda1c17b22796e53eb7ce3084a2af5de0ee68591b1e0fb WatchSource:0}: Error finding container 005df66637c9429378bda1c17b22796e53eb7ce3084a2af5de0ee68591b1e0fb: Status 404 returned error can't find the container with id 005df66637c9429378bda1c17b22796e53eb7ce3084a2af5de0ee68591b1e0fb
Sep 16 17:24:21.793617 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:21.793593    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71972a41c7eb6801726c8cf0011774d.slice/crio-d8be5d69ee7b52792e02ff997ccac587604a25af56691e8ce25a7ddd7002750e WatchSource:0}: Error finding container d8be5d69ee7b52792e02ff997ccac587604a25af56691e8ce25a7ddd7002750e: Status 404 returned error can't find the container with id d8be5d69ee7b52792e02ff997ccac587604a25af56691e8ce25a7ddd7002750e
Sep 16 17:24:21.798814 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.798794    2542 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:24:21.807575 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.807557    2542 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:21.838380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.838359    2542 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 17:24:21.839443 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.839427    2542 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal"
Sep 16 17:24:21.850195 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.850179    2542 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 17:24:21.901234 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:21.901212    2542 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:22.183563 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.183484    2542 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 17:24:22.383495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.383467    2542 apiserver.go:52] "Watching apiserver"
Sep 16 17:24:22.390483 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.390449    2542 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 17:24:22.390942 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.390915    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-ddjl5","openshift-multus/network-metrics-daemon-s76cv","openshift-dns/node-resolver-czk72","openshift-image-registry/node-ca-28mvz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal","openshift-multus/multus-7z6nl","openshift-network-diagnostics/network-check-target-kjfbw","openshift-network-operator/iptables-alerter-nvmqx","openshift-ovn-kubernetes/ovnkube-node-mrjcs","kube-system/global-pull-secret-syncer-7knvj","kube-system/konnectivity-agent-722wm","kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg","openshift-cluster-node-tuning-operator/tuned-6ndt4"]
Sep 16 17:24:22.393637 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.393614    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:22.393728 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.393691    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:22.395945 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.395923    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.398485 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.398417    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.400170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.400152    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 17:24:22.400286 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.400169    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kbrdz\""
Sep 16 17:24:22.400449 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.400425    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 17:24:22.400758 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.400740    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.401046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.401022    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.401299 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.401279    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.401512 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.401496    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pbxrr\""
Sep 16 17:24:22.403816 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.403252    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.405268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.405247    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 17:24:22.405582 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.405558    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.405697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.405628    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pvqfb\""
Sep 16 17:24:22.405850 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.405830    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.406740 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.406722    2542 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 17:24:22.406740 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.406736    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mpxnr\""
Sep 16 17:24:22.407747 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.407538    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.407860 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.407843    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.408334 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.408316    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.410480 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.410454    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.411804 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.411782    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4tk98\""
Sep 16 17:24:22.412018 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.411997    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 17:24:22.412143 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.412122    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 17:24:22.412852 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.412829    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.413491 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.413466    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.413938 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.413912    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 17:24:22.414248 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.414228    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 17:24:22.416497 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.414332    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.416497 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.415940    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.416497 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.416023    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:22.416820 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.416738    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.416820 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.416778    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 17:24:22.416987 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.416959    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.416987 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.416959    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 17:24:22.417099 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.416994    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mgz9t\""
Sep 16 17:24:22.418236 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418193    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pr5l4\" (UniqueName: \"kubernetes.io/projected/74e282fd-2c09-4e91-af47-b93ffce7077c-kube-api-access-pr5l4\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.418331 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418315    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-run\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.418402 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418358    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8282x\" (UniqueName: \"kubernetes.io/projected/794e229e-a830-4056-9682-2b72bcbf6eb4-kube-api-access-8282x\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.418402 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418392    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-multus\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418436    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-hostroot\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418463    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-multus-daemon-config\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418494    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-multus-certs\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418524    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49da429c-403a-46cf-a214-c584c906f98f-hosts-file\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.418682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418557    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74e282fd-2c09-4e91-af47-b93ffce7077c-serviceca\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.418682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418659    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-cnibin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418840 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418729    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-conf-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418840 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418773    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-etc-kubernetes\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.418965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418838    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63b6d3a8-e251-437b-95b6-853963dbe982-konnectivity-ca\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.419021 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418895    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-conf\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419021 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.418997    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-bin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419119 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419028    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419119 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419060    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-modprobe-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419119 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419090    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-sys\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419117    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-host\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419147    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-tmp\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419172    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-socket-dir-parent\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419199    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419226    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-cni-binary-copy\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419244    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-netns\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419265    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74e282fd-2c09-4e91-af47-b93ffce7077c-host\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419313    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-kubelet\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419334    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysconfig\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419353    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-kubernetes\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419371    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-tuned\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419397    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-system-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419435    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-k8s-cni-cncf-io\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419453    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv7cd\" (UniqueName: \"kubernetes.io/projected/49353fb8-5cda-4fc2-9660-8584d100b573-kube-api-access-nv7cd\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419471    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419486    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49da429c-403a-46cf-a214-c584c906f98f-tmp-dir\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419505    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlwr\" (UniqueName: \"kubernetes.io/projected/49da429c-403a-46cf-a214-c584c906f98f-kube-api-access-mhlwr\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419522    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-systemd\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419539    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-lib-modules\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419564    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-var-lib-kubelet\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.419619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419582    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63b6d3a8-e251-437b-95b6-853963dbe982-agent-certs\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.420349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.419981    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.420349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.420263    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.420349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.420302    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x99mg\""
Sep 16 17:24:22.421164 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.420584    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 17:24:22.421164 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.420752    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.421164 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.420812    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:22.424315 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.423745    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.424315 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.424287    2542 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 17:24:22.427280 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.427249    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 17:24:22.427505 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.427420    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 17:24:22.428181 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.428161    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-82pvq\""
Sep 16 17:24:22.428687 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.428667    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.431587 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.431567    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 17:24:22.431718 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.431672    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 17:24:22.431779 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.431674    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 17:24:22.432290 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.432257    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7gtfd\""
Sep 16 17:24:22.456165 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.456143    2542 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x58ss"
Sep 16 17:24:22.465061 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.465041    2542 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x58ss"
Sep 16 17:24:22.508879 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.508842    2542 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 17:24:22.519820 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519796    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-run\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.519960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519832    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-multus\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.519960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519856    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-hostroot\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.519960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519882    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-kubelet\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.519960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519908    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-cnibin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.519960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519944    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-conf-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.519974    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520001    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520026    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-socket-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520049    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-sys-fs\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520076    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63b6d3a8-e251-437b-95b6-853963dbe982-konnectivity-ca\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520101    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520137    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p87l4\" (UniqueName: \"kubernetes.io/projected/f313bc9f-a278-4e04-aaef-e2e9f30c18db-kube-api-access-p87l4\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520161    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-systemd-units\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520184    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-netns\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520208    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd660502-1d96-4943-b847-09265cc97f7f-ovn-node-metrics-cert\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520233    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-sys\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520256    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-host\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520282    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-socket-dir-parent\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520306    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-dbus\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520331    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520362    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520400    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520427    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-cni-binary-copy\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520463    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520492    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74e282fd-2c09-4e91-af47-b93ffce7077c-host\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520517    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2accaff6-c360-40b3-a18b-496141b4f9b7-host-slash\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520544    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520578    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-log-socket\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520604    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-registration-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520636    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-device-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520664    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysconfig\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.520746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520690    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-kubernetes\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520725    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-tuned\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520753    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520779    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djb4r\" (UniqueName: \"kubernetes.io/projected/2accaff6-c360-40b3-a18b-496141b4f9b7-kube-api-access-djb4r\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520805    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49da429c-403a-46cf-a214-c584c906f98f-tmp-dir\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520831    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-system-cni-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520858    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520883    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-config\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520908    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pr5l4\" (UniqueName: \"kubernetes.io/projected/74e282fd-2c09-4e91-af47-b93ffce7077c-kube-api-access-pr5l4\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520952    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8282x\" (UniqueName: \"kubernetes.io/projected/794e229e-a830-4056-9682-2b72bcbf6eb4-kube-api-access-8282x\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.520979    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-multus-daemon-config\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521004    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-multus-certs\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521029    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49da429c-403a-46cf-a214-c584c906f98f-hosts-file\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521052    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74e282fd-2c09-4e91-af47-b93ffce7077c-serviceca\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521089    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-etc-kubernetes\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521117    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-kubelet-config\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521143    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-node-log\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.521630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521168    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-bin\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521194    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-conf\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521221    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-bin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521246    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-systemd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521272    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-etc-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521306    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-ovn\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521351    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521380    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-modprobe-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521406    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-tmp\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521438    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521468    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-netns\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521496    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-netd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521526    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-script-lib\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521553    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-kubelet\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521579    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cnibin\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521608    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psrxk\" (UniqueName: \"kubernetes.io/projected/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kube-api-access-psrxk\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521636    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-var-lib-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.522495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521665    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-env-overrides\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521695    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqv9j\" (UniqueName: \"kubernetes.io/projected/dd660502-1d96-4943-b847-09265cc97f7f-kube-api-access-wqv9j\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521734    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-system-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521763    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-k8s-cni-cncf-io\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521792    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv7cd\" (UniqueName: \"kubernetes.io/projected/49353fb8-5cda-4fc2-9660-8584d100b573-kube-api-access-nv7cd\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521821    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521849    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-slash\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521878    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2accaff6-c360-40b3-a18b-496141b4f9b7-iptables-alerter-script\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521920    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlwr\" (UniqueName: \"kubernetes.io/projected/49da429c-403a-46cf-a214-c584c906f98f-kube-api-access-mhlwr\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.521973    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-systemd\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.522001    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-lib-modules\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.522055    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-var-lib-kubelet\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.522085    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f56vv\" (UniqueName: \"kubernetes.io/projected/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-kube-api-access-f56vv\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.522115    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.522192    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63b6d3a8-e251-437b-95b6-853963dbe982-agent-certs\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.523240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.523022    2542 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 17:24:22.524419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.524398    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-run\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.524506 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.524453    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-multus\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.524506 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.524494    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-hostroot\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.524684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.524558    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-cnibin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.524684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.524600    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-conf-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.525121    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-sys\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.525191    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-host\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.525242    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-socket-dir-parent\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.525360    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.525957    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-netns\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526022    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/63b6d3a8-e251-437b-95b6-853963dbe982-konnectivity-ca\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526091    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/74e282fd-2c09-4e91-af47-b93ffce7077c-host\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526192    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysconfig\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526274    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-kubernetes\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526698    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-multus-daemon-config\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526775    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-multus-certs\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.526970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.526824    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/49da429c-403a-46cf-a214-c584c906f98f-hosts-file\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.527609 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527229    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/74e282fd-2c09-4e91-af47-b93ffce7077c-serviceca\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.527609 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527450    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-etc-kubernetes\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.527609 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527584    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-sysctl-conf\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.527761 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527629    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-cni-bin\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.527761 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527755    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-multus-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.527860 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.527828    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-modprobe-d\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.528695 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.528662    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49da429c-403a-46cf-a214-c584c906f98f-tmp-dir\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.528820 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.528801    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-run-k8s-cni-cncf-io\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.528895 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.528875    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-host-var-lib-kubelet\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.529000 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.528985    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/49353fb8-5cda-4fc2-9660-8584d100b573-system-cni-dir\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.529533 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.529153    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-systemd\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.529533 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.529244    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-lib-modules\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.529533 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.529298    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794e229e-a830-4056-9682-2b72bcbf6eb4-var-lib-kubelet\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.529772 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.529555    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/49353fb8-5cda-4fc2-9660-8584d100b573-cni-binary-copy\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.530819 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.530797    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/63b6d3a8-e251-437b-95b6-853963dbe982-agent-certs\") pod \"konnectivity-agent-722wm\" (UID: \"63b6d3a8-e251-437b-95b6-853963dbe982\") " pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.531125 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.531106    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-etc-tuned\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.533063 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.533024    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/794e229e-a830-4056-9682-2b72bcbf6eb4-tmp\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.536747 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.536689    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pr5l4\" (UniqueName: \"kubernetes.io/projected/74e282fd-2c09-4e91-af47-b93ffce7077c-kube-api-access-pr5l4\") pod \"node-ca-28mvz\" (UID: \"74e282fd-2c09-4e91-af47-b93ffce7077c\") " pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.539107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.539086    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8282x\" (UniqueName: \"kubernetes.io/projected/794e229e-a830-4056-9682-2b72bcbf6eb4-kube-api-access-8282x\") pod \"tuned-6ndt4\" (UID: \"794e229e-a830-4056-9682-2b72bcbf6eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.543529 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.543506    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:22.543617 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.543539    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:22.543617 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.543554    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:22.543741 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.543625    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:23.043602763 +0000 UTC m=+3.035698378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:22.546474 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.546212    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv7cd\" (UniqueName: \"kubernetes.io/projected/49353fb8-5cda-4fc2-9660-8584d100b573-kube-api-access-nv7cd\") pod \"multus-7z6nl\" (UID: \"49353fb8-5cda-4fc2-9660-8584d100b573\") " pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.546797 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.546615    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlwr\" (UniqueName: \"kubernetes.io/projected/49da429c-403a-46cf-a214-c584c906f98f-kube-api-access-mhlwr\") pod \"node-resolver-czk72\" (UID: \"49da429c-403a-46cf-a214-c584c906f98f\") " pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.622525 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622478    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-kubelet-config\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.622525 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622519    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-node-log\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622543    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-bin\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622565    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-systemd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622587    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-etc-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622608    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-ovn\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622633    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622656    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-netd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622677    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-script-lib\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622701    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cnibin\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622727    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psrxk\" (UniqueName: \"kubernetes.io/projected/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kube-api-access-psrxk\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622728    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-kubelet-config\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.622749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622750    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-var-lib-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622775    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-env-overrides\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622786    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-node-log\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622802    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqv9j\" (UniqueName: \"kubernetes.io/projected/dd660502-1d96-4943-b847-09265cc97f7f-kube-api-access-wqv9j\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622845    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-slash\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622869    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2accaff6-c360-40b3-a18b-496141b4f9b7-iptables-alerter-script\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622901    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f56vv\" (UniqueName: \"kubernetes.io/projected/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-kube-api-access-f56vv\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622947    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622979    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-kubelet\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623005    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623030    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623055    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-socket-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623078    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-sys-fs\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623107    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623132    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p87l4\" (UniqueName: \"kubernetes.io/projected/f313bc9f-a278-4e04-aaef-e2e9f30c18db-kube-api-access-p87l4\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623158    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-systemd-units\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623189    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-netns\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.623311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623214    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd660502-1d96-4943-b847-09265cc97f7f-ovn-node-metrics-cert\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623244    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-dbus\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623268    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623296    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623326    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623354    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2accaff6-c360-40b3-a18b-496141b4f9b7-host-slash\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623378    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623403    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-log-socket\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623443    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-registration-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623459    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-script-lib\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623469    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-device-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623518    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623537    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-device-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623548    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djb4r\" (UniqueName: \"kubernetes.io/projected/2accaff6-c360-40b3-a18b-496141b4f9b7-kube-api-access-djb4r\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.622672    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-bin\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623576    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-system-cni-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623602    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.624200 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623600    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cnibin\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623642    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-config\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.623886    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-var-lib-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624151    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-ovnkube-config\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624212    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-systemd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624261    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-etc-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624305    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-ovn\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624307    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/dd660502-1d96-4943-b847-09265cc97f7f-env-overrides\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.624389    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.624462    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:23.12444266 +0000 UTC m=+3.116538256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624567    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-slash\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624670    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-cni-netd\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.624831    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.625096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625076    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-system-cni-dir\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625127    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2accaff6-c360-40b3-a18b-496141b4f9b7-iptables-alerter-script\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625182    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-etc-selinux\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625275    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-netns\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625323    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-socket-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625399    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625456    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-systemd-units\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625512    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-kubelet\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625564    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625630    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-sys-fs\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.625723    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:22.625778 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:22.625786    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:23.125768425 +0000 UTC m=+3.117864007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625839    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2accaff6-c360-40b3-a18b-496141b4f9b7-host-slash\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625894    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-run-openvswitch\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625875    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.625996    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/34a69a06-bfeb-4602-a1d9-dbd5181b884f-dbus\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.626025    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-log-socket\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.626109    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-registration-dir\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.626349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.626159    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dd660502-1d96-4943-b847-09265cc97f7f-host-run-ovn-kubernetes\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.626718 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.626554    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.626992 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.626967    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-cni-binary-copy\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.627960 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.627843    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal" event={"ID":"e71972a41c7eb6801726c8cf0011774d","Type":"ContainerStarted","Data":"d8be5d69ee7b52792e02ff997ccac587604a25af56691e8ce25a7ddd7002750e"}
Sep 16 17:24:22.628410 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.628385    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/dd660502-1d96-4943-b847-09265cc97f7f-ovn-node-metrics-cert\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.629331 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.629282    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal" event={"ID":"d95d3581c8d80d714714e1182983c2fc","Type":"ContainerStarted","Data":"005df66637c9429378bda1c17b22796e53eb7ce3084a2af5de0ee68591b1e0fb"}
Sep 16 17:24:22.638081 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.638030    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psrxk\" (UniqueName: \"kubernetes.io/projected/6df5def6-0b5f-47ac-a67c-dcc80b47d31a-kube-api-access-psrxk\") pod \"aws-ebs-csi-driver-node-dwrjg\" (UID: \"6df5def6-0b5f-47ac-a67c-dcc80b47d31a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.638081 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.638055    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djb4r\" (UniqueName: \"kubernetes.io/projected/2accaff6-c360-40b3-a18b-496141b4f9b7-kube-api-access-djb4r\") pod \"iptables-alerter-nvmqx\" (UID: \"2accaff6-c360-40b3-a18b-496141b4f9b7\") " pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.639105 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.639044    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p87l4\" (UniqueName: \"kubernetes.io/projected/f313bc9f-a278-4e04-aaef-e2e9f30c18db-kube-api-access-p87l4\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:22.640697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.640670    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqv9j\" (UniqueName: \"kubernetes.io/projected/dd660502-1d96-4943-b847-09265cc97f7f-kube-api-access-wqv9j\") pod \"ovnkube-node-mrjcs\" (UID: \"dd660502-1d96-4943-b847-09265cc97f7f\") " pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.641762 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.641722    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f56vv\" (UniqueName: \"kubernetes.io/projected/65383bc0-ebcb-4a48-87a1-fcaa1855f17e-kube-api-access-f56vv\") pod \"multus-additional-cni-plugins-ddjl5\" (UID: \"65383bc0-ebcb-4a48-87a1-fcaa1855f17e\") " pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.711349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.711276    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:22.721576 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.721556    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-czk72"
Sep 16 17:24:22.722052 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.722026    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b6d3a8_e251_437b_95b6_853963dbe982.slice/crio-f36e76d73d7a64b8cbbb65e74f9fd278b8321fe7db7d3ce1b6cd1c8d842180f8 WatchSource:0}: Error finding container f36e76d73d7a64b8cbbb65e74f9fd278b8321fe7db7d3ce1b6cd1c8d842180f8: Status 404 returned error can't find the container with id f36e76d73d7a64b8cbbb65e74f9fd278b8321fe7db7d3ce1b6cd1c8d842180f8
Sep 16 17:24:22.732481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.732449    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:22.733336 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.733305    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49da429c_403a_46cf_a214_c584c906f98f.slice/crio-7f17da2b278c9457b047cda6fc4f2a9cb1af858177ada1504b8df991503e6225 WatchSource:0}: Error finding container 7f17da2b278c9457b047cda6fc4f2a9cb1af858177ada1504b8df991503e6225: Status 404 returned error can't find the container with id 7f17da2b278c9457b047cda6fc4f2a9cb1af858177ada1504b8df991503e6225
Sep 16 17:24:22.738570 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.738405    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-28mvz"
Sep 16 17:24:22.743916 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.743892    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd660502_1d96_4943_b847_09265cc97f7f.slice/crio-746167199e8fc378577d05e3dbf1512db66bac50a9364c8037e71245ae9f0515 WatchSource:0}: Error finding container 746167199e8fc378577d05e3dbf1512db66bac50a9364c8037e71245ae9f0515: Status 404 returned error can't find the container with id 746167199e8fc378577d05e3dbf1512db66bac50a9364c8037e71245ae9f0515
Sep 16 17:24:22.746668 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.746642    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6ndt4"
Sep 16 17:24:22.749489 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.749455    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e282fd_2c09_4e91_af47_b93ffce7077c.slice/crio-3391324230f579961165ca47203e84381a98496264b500672a129cc6c83705b5 WatchSource:0}: Error finding container 3391324230f579961165ca47203e84381a98496264b500672a129cc6c83705b5: Status 404 returned error can't find the container with id 3391324230f579961165ca47203e84381a98496264b500672a129cc6c83705b5
Sep 16 17:24:22.756117 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.756094    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7z6nl"
Sep 16 17:24:22.756229 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.756200    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794e229e_a830_4056_9682_2b72bcbf6eb4.slice/crio-6ccce8b9eab5f7d20305510c46df1cecd89ea35fa7721eef1b66d090058245ae WatchSource:0}: Error finding container 6ccce8b9eab5f7d20305510c46df1cecd89ea35fa7721eef1b66d090058245ae: Status 404 returned error can't find the container with id 6ccce8b9eab5f7d20305510c46df1cecd89ea35fa7721eef1b66d090058245ae
Sep 16 17:24:22.764684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.764664    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg"
Sep 16 17:24:22.768005 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.767980    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49353fb8_5cda_4fc2_9660_8584d100b573.slice/crio-70b6665e7ddc38de4628eb30d9e1878a26add7ce566b22fd2b1f00f798ef1d36 WatchSource:0}: Error finding container 70b6665e7ddc38de4628eb30d9e1878a26add7ce566b22fd2b1f00f798ef1d36: Status 404 returned error can't find the container with id 70b6665e7ddc38de4628eb30d9e1878a26add7ce566b22fd2b1f00f798ef1d36
Sep 16 17:24:22.773526 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.773501    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ddjl5"
Sep 16 17:24:22.775062 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.774942    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df5def6_0b5f_47ac_a67c_dcc80b47d31a.slice/crio-f2467741624b1987dcb19704eac5ddb6278ac9e8d2cfed4fd877138ac3a49e81 WatchSource:0}: Error finding container f2467741624b1987dcb19704eac5ddb6278ac9e8d2cfed4fd877138ac3a49e81: Status 404 returned error can't find the container with id f2467741624b1987dcb19704eac5ddb6278ac9e8d2cfed4fd877138ac3a49e81
Sep 16 17:24:22.781086 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:22.780920    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nvmqx"
Sep 16 17:24:22.784968 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.784943    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65383bc0_ebcb_4a48_87a1_fcaa1855f17e.slice/crio-3d01d71f68f851b7c4f4d1c1e99d6a440d73ec64cfd702c087518620540a0f17 WatchSource:0}: Error finding container 3d01d71f68f851b7c4f4d1c1e99d6a440d73ec64cfd702c087518620540a0f17: Status 404 returned error can't find the container with id 3d01d71f68f851b7c4f4d1c1e99d6a440d73ec64cfd702c087518620540a0f17
Sep 16 17:24:22.793951 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:22.793919    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2accaff6_c360_40b3_a18b_496141b4f9b7.slice/crio-5725a0a02d55d4be0152c7ef591a75630a31750da01d7d3e2801000e397e81b1 WatchSource:0}: Error finding container 5725a0a02d55d4be0152c7ef591a75630a31750da01d7d3e2801000e397e81b1: Status 404 returned error can't find the container with id 5725a0a02d55d4be0152c7ef591a75630a31750da01d7d3e2801000e397e81b1
Sep 16 17:24:23.127996 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.127904    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:23.127996 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.127957    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.128018    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128124    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128140    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128148    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128168    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128204    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:24.128183089 +0000 UTC m=+4.120278679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128222    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:24.128215222 +0000 UTC m=+4.120310798 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:23.128245 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128139    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:23.128602 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.128294    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:24.128269659 +0000 UTC m=+4.120365247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:23.469734 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.469589    2542 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 17:19:22 +0000 UTC" deadline="2028-02-19 05:56:16.009206429 +0000 UTC"
Sep 16 17:24:23.469734 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.469630    2542 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12492h31m52.539580664s"
Sep 16 17:24:23.618316 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.618249    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:23.618507 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.618376    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:23.618795 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.618624    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:23.618795 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.618249    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:23.618795 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.618752    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:23.619042 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:23.618802    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:23.639961 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.639911    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nvmqx" event={"ID":"2accaff6-c360-40b3-a18b-496141b4f9b7","Type":"ContainerStarted","Data":"5725a0a02d55d4be0152c7ef591a75630a31750da01d7d3e2801000e397e81b1"}
Sep 16 17:24:23.642367 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.642316    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z6nl" event={"ID":"49353fb8-5cda-4fc2-9660-8584d100b573","Type":"ContainerStarted","Data":"70b6665e7ddc38de4628eb30d9e1878a26add7ce566b22fd2b1f00f798ef1d36"}
Sep 16 17:24:23.646655 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.646561    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"746167199e8fc378577d05e3dbf1512db66bac50a9364c8037e71245ae9f0515"}
Sep 16 17:24:23.648104 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.648031    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-czk72" event={"ID":"49da429c-403a-46cf-a214-c584c906f98f","Type":"ContainerStarted","Data":"7f17da2b278c9457b047cda6fc4f2a9cb1af858177ada1504b8df991503e6225"}
Sep 16 17:24:23.656067 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.656037    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-722wm" event={"ID":"63b6d3a8-e251-437b-95b6-853963dbe982","Type":"ContainerStarted","Data":"f36e76d73d7a64b8cbbb65e74f9fd278b8321fe7db7d3ce1b6cd1c8d842180f8"}
Sep 16 17:24:23.662333 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.662285    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerStarted","Data":"3d01d71f68f851b7c4f4d1c1e99d6a440d73ec64cfd702c087518620540a0f17"}
Sep 16 17:24:23.663976 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.663936    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg" event={"ID":"6df5def6-0b5f-47ac-a67c-dcc80b47d31a","Type":"ContainerStarted","Data":"f2467741624b1987dcb19704eac5ddb6278ac9e8d2cfed4fd877138ac3a49e81"}
Sep 16 17:24:23.665731 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.665672    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6ndt4" event={"ID":"794e229e-a830-4056-9682-2b72bcbf6eb4","Type":"ContainerStarted","Data":"6ccce8b9eab5f7d20305510c46df1cecd89ea35fa7721eef1b66d090058245ae"}
Sep 16 17:24:23.667852 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:23.667800    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-28mvz" event={"ID":"74e282fd-2c09-4e91-af47-b93ffce7077c","Type":"ContainerStarted","Data":"3391324230f579961165ca47203e84381a98496264b500672a129cc6c83705b5"}
Sep 16 17:24:24.137395 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:24.137255    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:24.137395 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:24.137353    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:24.137408    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137525    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137577    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137593    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137614    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:24.137650 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137620    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:26.137597968 +0000 UTC m=+6.129693549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:24.138016 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137661    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:26.137649011 +0000 UTC m=+6.129744586 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:24.138016 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137760    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:24.138016 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:24.137786    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:26.137777516 +0000 UTC m=+6.129873092 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:24.469994 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:24.469843    2542 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 17:19:22 +0000 UTC" deadline="2028-02-21 16:58:49.078359807 +0000 UTC"
Sep 16 17:24:24.469994 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:24.469929    2542 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12551h34m24.608436348s"
Sep 16 17:24:25.618588 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:25.618559    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:25.619121 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:25.618693    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:25.619184 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:25.619128    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:25.619267 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:25.619242    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:25.619338 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:25.619319    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:25.619415 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:25.619395    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:26.154076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:26.154039    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:26.154274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:26.154102    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:26.154274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:26.154135    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:26.154397 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154281    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:26.154397 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154300    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:26.154397 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154313    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:26.154532 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154475    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:30.154453195 +0000 UTC m=+10.146548793 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:26.154915 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154892    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:26.155060 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.154965    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:30.154948133 +0000 UTC m=+10.147043708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:26.155060 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.155025    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:26.155060 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:26.155061    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:30.155048977 +0000 UTC m=+10.147144553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:27.617819    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:27.617971    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:27.618377    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:27.618493    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:27.618543    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:27.618682 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:27.618619    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:29.618072 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:29.618037    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:29.618624 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:29.618085    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:29.618624 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:29.618085    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:29.618624 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:29.618166    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:29.618624 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:29.618283    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:29.618624 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:29.618371    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:30.188378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:30.188211    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:30.188378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:30.188256    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:30.188378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:30.188298    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188401    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188421    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188434    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188439    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188486    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:38.188469001 +0000 UTC m=+18.180564590 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188506    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:38.188497589 +0000 UTC m=+18.180593180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188505    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:30.188667 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:30.188562    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:38.188543084 +0000 UTC m=+18.180638662 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:31.617867 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:31.617803    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:31.618332 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:31.617969    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:31.618332 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:31.618305    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:31.618442 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:31.618387    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:31.618442 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:31.618410    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:31.618534 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:31.618490    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:33.618354 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:33.618319    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:33.618729 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:33.618319    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:33.618729 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:33.618439    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:33.618729 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:33.618553    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:33.618729 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:33.618320    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:33.618729 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:33.618653    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:35.617928 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:35.617891    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:35.618358 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:35.617891    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:35.618358 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:35.618009    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:35.618358 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:35.617891    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:35.618358 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:35.618093    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:35.618358 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:35.618186    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:37.618619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:37.618580    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:37.619177 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:37.618580    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:37.619177 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:37.618700    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:37.619177 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:37.618782    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:37.619177 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:37.618580    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:37.619177 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:37.618951    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:38.249383 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:38.249348    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:38.249543 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:38.249402    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:38.249543 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249491    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:38.249543 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249493    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:38.249543 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:38.249525    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:38.249543 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249545    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:24:54.249525218 +0000 UTC m=+34.241620803 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:38.249732 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249610    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:38.249732 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249624    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:24:54.249607518 +0000 UTC m=+34.241703101 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:38.249732 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249627    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:38.249732 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249640    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:38.249732 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:38.249682    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:54.249671698 +0000 UTC m=+34.241767302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:39.617868 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:39.617801    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:39.618368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:39.617878    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:39.618368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:39.617880    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:39.618368 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:39.617961    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:39.618368 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:39.618060    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:39.618368 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:39.618131    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.618046    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:41.618577    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.618661    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:41.618744    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.618798    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:41.618941 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:41.618891    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:41.713000 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.712970    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7z6nl" event={"ID":"49353fb8-5cda-4fc2-9660-8584d100b573","Type":"ContainerStarted","Data":"3be8804e0571f88939948c29f3fa1e5cca84c889e306e6e802df5d7d8a3664fb"}
Sep 16 17:24:41.715272 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715254    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:24:41.715556 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715538    2542 generic.go:356] "Generic (PLEG): container finished" podID="dd660502-1d96-4943-b847-09265cc97f7f" containerID="b03be8d3cc6cde81a6463943455c86b64bed0be8701cf388f06e4178650ddc20" exitCode=1
Sep 16 17:24:41.715670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715607    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"113c9c66b4d878b57face7e0ec969b4fc014ae4450d49f51c73a2c52b7150aae"}
Sep 16 17:24:41.715791 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715774    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"cc43b55a23e0ea95ff156fbf286c2daa7fe46dff116f842394e67a9c24696aac"}
Sep 16 17:24:41.715871 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715832    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"8729fc9375e80db1638bc5cc6b5375b3ea3fe6a0327fb9a8401e1be289a8193f"}
Sep 16 17:24:41.715923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715900    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerDied","Data":"b03be8d3cc6cde81a6463943455c86b64bed0be8701cf388f06e4178650ddc20"}
Sep 16 17:24:41.715969 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.715952    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"4374579d926f4ee25d8def99c8dd2f21b13d75fa8668a4426a36c0fb24198601"}
Sep 16 17:24:41.717105 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.717085    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6ndt4" event={"ID":"794e229e-a830-4056-9682-2b72bcbf6eb4","Type":"ContainerStarted","Data":"73e3decb021238311fa9ad15b28ac5c2502c8633580fcfd237df87414b9b9009"}
Sep 16 17:24:41.718488 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.718470    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal" event={"ID":"d95d3581c8d80d714714e1182983c2fc","Type":"ContainerStarted","Data":"113e07a88e125f6d3f3d2b1d055014f635199796c68544e4b347c01263f3db70"}
Sep 16 17:24:41.744017 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.743973    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7z6nl" podStartSLOduration=2.460933124 podStartE2EDuration="20.743960835s" totalImagesPullingTime="18.283027711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:41.72935527 +0000 UTC m=+21.721450865" watchObservedRunningTime="2026-09-16 17:24:41.743960835 +0000 UTC m=+21.736056408"
Sep 16 17:24:41.744412 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.744386    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6ndt4" podStartSLOduration=2.484298511 podStartE2EDuration="20.744381014s" totalImagesPullingTime="18.260082503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:41.744076016 +0000 UTC m=+21.736171611" watchObservedRunningTime="2026-09-16 17:24:41.744381014 +0000 UTC m=+21.736476669"
Sep 16 17:24:41.758107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:41.758068    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-96.ec2.internal" podStartSLOduration=19.758057938 podStartE2EDuration="19.758057938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:41.757940065 +0000 UTC m=+21.750035674" watchObservedRunningTime="2026-09-16 17:24:41.758057938 +0000 UTC m=+21.750153534"
Sep 16 17:24:42.722684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.722585    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nvmqx" event={"ID":"2accaff6-c360-40b3-a18b-496141b4f9b7","Type":"ContainerStarted","Data":"66bf5d0fea63956e9c8b650f1756d1ac480dfcecf8c67ec98e2860ab012ee603"}
Sep 16 17:24:42.726110 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.726079    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:24:42.726558 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.726533    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"bef39a5335803ee9bc1e109c14929d710a8c1dd6bdaeb6683a3e72e01aefbcc2"}
Sep 16 17:24:42.728156 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.728132    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-czk72" event={"ID":"49da429c-403a-46cf-a214-c584c906f98f","Type":"ContainerStarted","Data":"3e8b2a530ae94bd4e8ac8a70453a465b525a752efac4847247bf4550c81d8669"}
Sep 16 17:24:42.729738 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.729716    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-722wm" event={"ID":"63b6d3a8-e251-437b-95b6-853963dbe982","Type":"ContainerStarted","Data":"866e5bc767a40a990f6ac45173220b1d46783ad4a2e4d9c84485fca4771ec1d2"}
Sep 16 17:24:42.731772 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.731751    2542 generic.go:356] "Generic (PLEG): container finished" podID="e71972a41c7eb6801726c8cf0011774d" containerID="e40f310c799c9746146496551a528b1e00648081d4878b70590dc9ef242118cc" exitCode=0
Sep 16 17:24:42.731874 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.731823    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal" event={"ID":"e71972a41c7eb6801726c8cf0011774d","Type":"ContainerDied","Data":"e40f310c799c9746146496551a528b1e00648081d4878b70590dc9ef242118cc"}
Sep 16 17:24:42.733475 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.733455    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="cd6f908dab98e98a5de71260bd70b273ba3699f958be082e802b9364cffaf982" exitCode=0
Sep 16 17:24:42.733555 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.733523    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"cd6f908dab98e98a5de71260bd70b273ba3699f958be082e802b9364cffaf982"}
Sep 16 17:24:42.735318 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.735298    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg" event={"ID":"6df5def6-0b5f-47ac-a67c-dcc80b47d31a","Type":"ContainerStarted","Data":"65df2ea9ca2cea3a813e656f1a0146427d6c48a91e09520809a3e311fc3ef3f1"}
Sep 16 17:24:42.736977 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.736952    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-28mvz" event={"ID":"74e282fd-2c09-4e91-af47-b93ffce7077c","Type":"ContainerStarted","Data":"10d462a92cbe92615558ce845b18b972616c6ff186afe60c7da586801c36b406"}
Sep 16 17:24:42.738413 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.738368    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nvmqx" podStartSLOduration=3.514676468 podStartE2EDuration="21.73835245s" totalImagesPullingTime="18.223675982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:42.738338982 +0000 UTC m=+22.730434578" watchObservedRunningTime="2026-09-16 17:24:42.73835245 +0000 UTC m=+22.730448048"
Sep 16 17:24:42.787659 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.787597    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-722wm" podStartSLOduration=3.504066766 podStartE2EDuration="21.787583443s" totalImagesPullingTime="18.283516677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:42.76889649 +0000 UTC m=+22.760992089" watchObservedRunningTime="2026-09-16 17:24:42.787583443 +0000 UTC m=+22.779679037"
Sep 16 17:24:42.788033 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.787996    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-28mvz" podStartSLOduration=3.532839326 podStartE2EDuration="21.787986722s" totalImagesPullingTime="18.255147396s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:42.787776598 +0000 UTC m=+22.779872192" watchObservedRunningTime="2026-09-16 17:24:42.787986722 +0000 UTC m=+22.780082319"
Sep 16 17:24:42.821983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.821939    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-czk72" podStartSLOduration=3.5402157340000002 podStartE2EDuration="21.821923085s" totalImagesPullingTime="18.281707351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:42.821542636 +0000 UTC m=+22.813638231" watchObservedRunningTime="2026-09-16 17:24:42.821923085 +0000 UTC m=+22.814018682"
Sep 16 17:24:42.959135 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:42.959113    2542 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 17:24:43.618392 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.618367    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:43.618631 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.618455    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:43.618631 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:43.618462    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:43.618631 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.618486    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:43.618631 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:43.618567    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:43.618770 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:43.618633    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:43.620434 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.620368    2542 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T17:24:42.959131175Z","UUID":"6a8d5d18-70be-43ae-ab33-8b263d0e1978","Handler":null,"Name":"","Endpoint":""}
Sep 16 17:24:43.621860 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.621810    2542 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 17:24:43.621986 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.621875    2542 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 17:24:43.741070 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.741031    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal" event={"ID":"e71972a41c7eb6801726c8cf0011774d","Type":"ContainerStarted","Data":"8a8146a96da7f7d0827cc836b3a5aa7d6e110974e2cbb8db939a3f65a589d6b2"}
Sep 16 17:24:43.743215 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:43.743181    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg" event={"ID":"6df5def6-0b5f-47ac-a67c-dcc80b47d31a","Type":"ContainerStarted","Data":"31fa6b463cb00a53a9a6706b350e3cc1e5d73ffbad10d5784c4664e0306704ce"}
Sep 16 17:24:44.748177 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:44.747929    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg" event={"ID":"6df5def6-0b5f-47ac-a67c-dcc80b47d31a","Type":"ContainerStarted","Data":"86f827d32933f247642c6190348bbcf262b2703f629b051c5857d09328e3c532"}
Sep 16 17:24:44.751709 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:44.751681    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:24:44.752245 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:44.752224    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"4115b179010d9207a5c808e65b2bc945c75334ce2b64f280bf4c9c5ef6d7b41e"}
Sep 16 17:24:44.771299 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:44.771255    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-96.ec2.internal" podStartSLOduration=22.771244101 podStartE2EDuration="22.771244101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:43.790435722 +0000 UTC m=+23.782531319" watchObservedRunningTime="2026-09-16 17:24:44.771244101 +0000 UTC m=+24.763339695"
Sep 16 17:24:45.618649 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:45.618614    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:45.618834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:45.618719    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:45.618834 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:45.618732    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:45.618834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:45.618744    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:45.618834 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:45.618812    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:45.619072 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:45.618892    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:46.668818 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:46.668785    2542 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:46.669647 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:46.669628    2542 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:46.707894 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:46.707830    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dwrjg" podStartSLOduration=5.739451398 podStartE2EDuration="25.70781236s" totalImagesPullingTime="19.968360962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:44.774978137 +0000 UTC m=+24.767073732" watchObservedRunningTime="2026-09-16 17:24:46.70781236 +0000 UTC m=+26.699907955"
Sep 16 17:24:46.756658 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:46.756631    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:46.757168 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:46.757144    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-722wm"
Sep 16 17:24:47.618394 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.618372    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:47.618491 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.618412    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:47.618491 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.618412    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:47.618585 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:47.618489    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:47.618585 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:47.618570    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:47.618649 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:47.618628    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:47.763242 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.763219    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:24:47.763616 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.763578    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"9d5449aa500dcb0d1c5ad8c9cc71be7ca304f8a1ccffcb693ccf80bd03b9fd7f"}
Sep 16 17:24:47.763987 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.763882    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:47.763987 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.763939    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:47.764152 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.764043    2542 scope.go:117] "RemoveContainer" containerID="b03be8d3cc6cde81a6463943455c86b64bed0be8701cf388f06e4178650ddc20"
Sep 16 17:24:47.779320 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:47.779266    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:48.768995 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.768770    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:24:48.769405 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.769263    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" event={"ID":"dd660502-1d96-4943-b847-09265cc97f7f","Type":"ContainerStarted","Data":"82a5e18e7196334f77d03c224132285e6ae8af480e38e7243739b327d345558e"}
Sep 16 17:24:48.769483 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.769468    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:48.771234 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.771212    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="5fa1b6314daee020033714c1f670331d3ab6053b430787e741c229a5895f79b2" exitCode=0
Sep 16 17:24:48.771368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.771302    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"5fa1b6314daee020033714c1f670331d3ab6053b430787e741c229a5895f79b2"}
Sep 16 17:24:48.784247 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.784229    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:24:48.841531 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:48.841479    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs" podStartSLOduration=9.346020007 podStartE2EDuration="27.841460663s" totalImagesPullingTime="18.495440656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:48.84094083 +0000 UTC m=+28.833036511" watchObservedRunningTime="2026-09-16 17:24:48.841460663 +0000 UTC m=+28.833556258"
Sep 16 17:24:49.618045 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:49.618012    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:49.618205 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:49.618119    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:49.618205 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:49.618134    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:49.618205 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:49.618146    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:49.618368 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:49.618205    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:49.618368 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:49.618292    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:50.035669 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.035635    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s76cv"]
Sep 16 17:24:50.036199 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.035804    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:50.036199 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:50.035945    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:50.036708 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.036691    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kjfbw"]
Sep 16 17:24:50.036793 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.036781    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:50.036913 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:50.036893    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:50.037295 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.037273    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7knvj"]
Sep 16 17:24:50.037372 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.037352    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:50.037451 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:50.037435    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:50.778251 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.778218    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="bbd745423c26b0119a1da366d0df30f3e0cf82c597fcc28d7772b0dfd23fc4ec" exitCode=0
Sep 16 17:24:50.778399 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:50.778315    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"bbd745423c26b0119a1da366d0df30f3e0cf82c597fcc28d7772b0dfd23fc4ec"}
Sep 16 17:24:51.618293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:51.618217    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:51.618293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:51.618249    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:51.618293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:51.618249    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:51.618769 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:51.618323    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:51.618769 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:51.618401    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:51.618769 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:51.618475    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:51.783026 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:51.782989    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="ad4a2068cc6470dd99b53b5270ceeafe375b866f5b0909e09084996db16929e4" exitCode=0
Sep 16 17:24:51.783212 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:51.783043    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"ad4a2068cc6470dd99b53b5270ceeafe375b866f5b0909e09084996db16929e4"}
Sep 16 17:24:53.617748 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:53.617707    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:53.618222 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:53.617810    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:53.618222 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:53.617818    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kjfbw" podUID="42df8adb-ae13-421d-a547-3b5a11d49657"
Sep 16 17:24:53.618222 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:53.617867    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:53.618222 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:53.617996    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:24:53.618222 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:53.618086    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7knvj" podUID="34a69a06-bfeb-4602-a1d9-dbd5181b884f"
Sep 16 17:24:54.267399 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.267363    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:54.267565 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.267431    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:54.267565 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.267463    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:54.267565 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267544    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:54.267565 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267561    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267580    2542 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267589    2542 projected.go:196] Error preparing data for projected volume kube-api-access-rfkxg for pod openshift-network-diagnostics/network-check-target-kjfbw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267612    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.267597442 +0000 UTC m=+66.259693029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267634    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg podName:42df8adb-ae13-421d-a547-3b5a11d49657 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.267620313 +0000 UTC m=+66.259715886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rfkxg" (UniqueName: "kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg") pod "network-check-target-kjfbw" (UID: "42df8adb-ae13-421d-a547-3b5a11d49657") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267544    2542 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:54.267724 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.267672    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret podName:34a69a06-bfeb-4602-a1d9-dbd5181b884f nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.26766516 +0000 UTC m=+66.259760733 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret") pod "global-pull-secret-syncer-7knvj" (UID: "34a69a06-bfeb-4602-a1d9-dbd5181b884f") : object "kube-system"/"original-pull-secret" not registered
Sep 16 17:24:54.269237 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.269214    2542 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-96.ec2.internal" event="NodeReady"
Sep 16 17:24:54.269395 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.269345    2542 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 17:24:54.309541 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.309514    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84c4f75687-wklww"]
Sep 16 17:24:54.314025 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.313593    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.314553 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.314528    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"]
Sep 16 17:24:54.318043 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.318023    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"]
Sep 16 17:24:54.318148 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.318135    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.321894 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.321774    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"]
Sep 16 17:24:54.322198 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.322180    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.323834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.323809    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 17:24:54.323998 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.323899    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-jdz5h\""
Sep 16 17:24:54.324120 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.324104    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 17:24:54.324181 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.324147    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 17:24:54.324237 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.324189    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 17:24:54.324237 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.324195    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 17:24:54.326124 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.326104    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.326218 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.326206    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 17:24:54.326817 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.326796    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 17:24:54.327310 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.327293    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 17:24:54.327437 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.327419    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 16 17:24:54.327836 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.327819    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 17:24:54.328330 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.328146    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45pjq\""
Sep 16 17:24:54.329009 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.328962    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 17:24:54.330547 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.330518    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 16 17:24:54.333461 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.332433    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84c4f75687-wklww"]
Sep 16 17:24:54.333461 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.332872    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 17:24:54.333461 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.333324    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rkb45"]
Sep 16 17:24:54.337025 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.337006    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5d5hd"]
Sep 16 17:24:54.337232 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.337211    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.339682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.339664    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 17:24:54.340406 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.340389    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"]
Sep 16 17:24:54.340503 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.340451    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"]
Sep 16 17:24:54.340503 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.340470    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"]
Sep 16 17:24:54.340603 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.340504    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.341122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.340891    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v8tmk\""
Sep 16 17:24:54.341122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.341019    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 17:24:54.344019 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.343997    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 17:24:54.344956 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.344929    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 17:24:54.345256 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.345210    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 17:24:54.345356 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.345331    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nqgn2\""
Sep 16 17:24:54.352812 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.352795    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rkb45"]
Sep 16 17:24:54.370667 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.370647    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5d5hd"]
Sep 16 17:24:54.469050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.468978    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469010    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469031    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d92892f1-d976-4c50-ab9d-0398376fb6a1-config-volume\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469058    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9965d588-9866-4a8c-8e44-ff2596388671-tmp\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469075    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9965d588-9866-4a8c-8e44-ff2596388671-klusterlet-config\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469089    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwxdv\" (UniqueName: \"kubernetes.io/projected/9965d588-9866-4a8c-8e44-ff2596388671-kube-api-access-pwxdv\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469104    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92892f1-d976-4c50-ab9d-0398376fb6a1-tmp-dir\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469122    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469140    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql55h\" (UniqueName: \"kubernetes.io/projected/4d0a280d-1f96-417a-bbf1-4f083f70ec10-kube-api-access-ql55h\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469154    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469169    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469185    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60d86da6-1f33-439b-9ee5-a289ed6c00e8-cert\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469222    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469258 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469244    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469284    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469359    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtzd\" (UniqueName: \"kubernetes.io/projected/60d86da6-1f33-439b-9ee5-a289ed6c00e8-kube-api-access-vgtzd\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469417    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb7f9\" (UniqueName: \"kubernetes.io/projected/d92892f1-d976-4c50-ab9d-0398376fb6a1-kube-api-access-fb7f9\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469478    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d0a280d-1f96-417a-bbf1-4f083f70ec10-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469506    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6v2q\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469562    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469598    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/417fecd4-200c-4610-bc72-a17ea41bb2af-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469628    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qzg8\" (UniqueName: \"kubernetes.io/projected/417fecd4-200c-4610-bc72-a17ea41bb2af-kube-api-access-2qzg8\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469651    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469680    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.469767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.469723    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.570810 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570779    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.570810 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570813    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570838    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570891    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570918    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d92892f1-d976-4c50-ab9d-0398376fb6a1-config-volume\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570952    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9965d588-9866-4a8c-8e44-ff2596388671-tmp\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.570960    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.570979    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:24:54.571055 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.571050    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:24:55.071029158 +0000 UTC m=+35.063124755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:24:54.571350 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.570976    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9965d588-9866-4a8c-8e44-ff2596388671-klusterlet-config\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.571402 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571375    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwxdv\" (UniqueName: \"kubernetes.io/projected/9965d588-9866-4a8c-8e44-ff2596388671-kube-api-access-pwxdv\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.571437 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571404    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92892f1-d976-4c50-ab9d-0398376fb6a1-tmp-dir\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.571437 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571420    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9965d588-9866-4a8c-8e44-ff2596388671-tmp\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.571494 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571436    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571494 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571466    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql55h\" (UniqueName: \"kubernetes.io/projected/4d0a280d-1f96-417a-bbf1-4f083f70ec10-kube-api-access-ql55h\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.571552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571490    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.571552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571515    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571531    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d92892f1-d976-4c50-ab9d-0398376fb6a1-config-volume\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.571552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571542    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60d86da6-1f33-439b-9ee5-a289ed6c00e8-cert\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571596    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571624    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571656    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571671    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtzd\" (UniqueName: \"kubernetes.io/projected/60d86da6-1f33-439b-9ee5-a289ed6c00e8-kube-api-access-vgtzd\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571694    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb7f9\" (UniqueName: \"kubernetes.io/projected/d92892f1-d976-4c50-ab9d-0398376fb6a1-kube-api-access-fb7f9\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571724    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d0a280d-1f96-417a-bbf1-4f083f70ec10-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.571749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571747    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6v2q\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.572078 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571788    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.572078 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.571817    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/417fecd4-200c-4610-bc72-a17ea41bb2af-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.572943    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qzg8\" (UniqueName: \"kubernetes.io/projected/417fecd4-200c-4610-bc72-a17ea41bb2af-kube-api-access-2qzg8\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.572996    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.573244    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d92892f1-d976-4c50-ab9d-0398376fb6a1-tmp-dir\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.573421    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.575375    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.575451    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60d86da6-1f33-439b-9ee5-a289ed6c00e8-cert\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.575521    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.575769    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.576269    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.576383    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.576548    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-hub\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.576653    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:54.576722    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:55.076704973 +0000 UTC m=+35.068800560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.577244    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/417fecd4-200c-4610-bc72-a17ea41bb2af-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.578148    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.579050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.578393    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/417fecd4-200c-4610-bc72-a17ea41bb2af-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.579943 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.579131    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4d0a280d-1f96-417a-bbf1-4f083f70ec10-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.580807 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.580788    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9965d588-9866-4a8c-8e44-ff2596388671-klusterlet-config\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.591958 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.591933    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6v2q\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.592197 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.592176    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwxdv\" (UniqueName: \"kubernetes.io/projected/9965d588-9866-4a8c-8e44-ff2596388671-kube-api-access-pwxdv\") pod \"klusterlet-addon-workmgr-74dfcfb986-rrhlx\" (UID: \"9965d588-9866-4a8c-8e44-ff2596388671\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.592619 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.592596    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql55h\" (UniqueName: \"kubernetes.io/projected/4d0a280d-1f96-417a-bbf1-4f083f70ec10-kube-api-access-ql55h\") pod \"managed-serviceaccount-addon-agent-95b756b44-2jkn6\" (UID: \"4d0a280d-1f96-417a-bbf1-4f083f70ec10\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.593709 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.593677    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb7f9\" (UniqueName: \"kubernetes.io/projected/d92892f1-d976-4c50-ab9d-0398376fb6a1-kube-api-access-fb7f9\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:54.595226 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.595207    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtzd\" (UniqueName: \"kubernetes.io/projected/60d86da6-1f33-439b-9ee5-a289ed6c00e8-kube-api-access-vgtzd\") pod \"ingress-canary-5d5hd\" (UID: \"60d86da6-1f33-439b-9ee5-a289ed6c00e8\") " pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:54.596019 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.596001    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:54.596105 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.596091    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qzg8\" (UniqueName: \"kubernetes.io/projected/417fecd4-200c-4610-bc72-a17ea41bb2af-kube-api-access-2qzg8\") pod \"cluster-proxy-proxy-agent-6c6f77989d-pg6gk\" (UID: \"417fecd4-200c-4610-bc72-a17ea41bb2af\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.642867 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.642821    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"
Sep 16 17:24:54.651107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.651069    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:24:54.669148 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.669129    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:24:54.681405 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:54.681386    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5d5hd"
Sep 16 17:24:55.077811 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.077774    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:55.078006 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.077832    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:55.078006 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:55.077969    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:24:55.078117 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:55.078024    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:24:55.078117 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:55.078039    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:56.078018082 +0000 UTC m=+36.070113658 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:24:55.078117 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:55.078043    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:24:55.078117 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:55.078100    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:24:56.078080522 +0000 UTC m=+36.070176119 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:24:55.618732 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.618606    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:24:55.618839 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.618607    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:24:55.618917 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.618612    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:24:55.624424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.624405    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 17:24:55.625324 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.625303    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 17:24:55.625397 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.625338    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2lh24\""
Sep 16 17:24:55.625397 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.625307    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 17:24:55.625631 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.625615    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 17:24:55.625700 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.625637    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q452j\""
Sep 16 17:24:55.753013 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.752989    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6"]
Sep 16 17:24:55.753940 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.753925    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5d5hd"]
Sep 16 17:24:55.754814 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.754791    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"]
Sep 16 17:24:55.761120 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.761102    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"]
Sep 16 17:24:55.777242 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:55.777220    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d0a280d_1f96_417a_bbf1_4f083f70ec10.slice/crio-9bc480f780b29386fb88e30758504eba313cdada3e0a7f16b5613733cb23c852 WatchSource:0}: Error finding container 9bc480f780b29386fb88e30758504eba313cdada3e0a7f16b5613733cb23c852: Status 404 returned error can't find the container with id 9bc480f780b29386fb88e30758504eba313cdada3e0a7f16b5613733cb23c852
Sep 16 17:24:55.786045 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:55.786018    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d86da6_1f33_439b_9ee5_a289ed6c00e8.slice/crio-db5e97a2e101367f4d14ce667c7cf0faec8e5d1dd6e61d48f1be136c93bec365 WatchSource:0}: Error finding container db5e97a2e101367f4d14ce667c7cf0faec8e5d1dd6e61d48f1be136c93bec365: Status 404 returned error can't find the container with id db5e97a2e101367f4d14ce667c7cf0faec8e5d1dd6e61d48f1be136c93bec365
Sep 16 17:24:55.786381 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:55.786358    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9965d588_9866_4a8c_8e44_ff2596388671.slice/crio-fbda3e566dbf155d5a01a6f25178120668544b1ef4fe46cb9e03a1652e6da481 WatchSource:0}: Error finding container fbda3e566dbf155d5a01a6f25178120668544b1ef4fe46cb9e03a1652e6da481: Status 404 returned error can't find the container with id fbda3e566dbf155d5a01a6f25178120668544b1ef4fe46cb9e03a1652e6da481
Sep 16 17:24:55.786602 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:24:55.786582    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod417fecd4_200c_4610_bc72_a17ea41bb2af.slice/crio-eaf0c0591367eba010d94f00da35a19ea0336ba28479ac4abf4afc803ba112c1 WatchSource:0}: Error finding container eaf0c0591367eba010d94f00da35a19ea0336ba28479ac4abf4afc803ba112c1: Status 404 returned error can't find the container with id eaf0c0591367eba010d94f00da35a19ea0336ba28479ac4abf4afc803ba112c1
Sep 16 17:24:55.794270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.794179    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5d5hd" event={"ID":"60d86da6-1f33-439b-9ee5-a289ed6c00e8","Type":"ContainerStarted","Data":"db5e97a2e101367f4d14ce667c7cf0faec8e5d1dd6e61d48f1be136c93bec365"}
Sep 16 17:24:55.795331 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.795301    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" event={"ID":"4d0a280d-1f96-417a-bbf1-4f083f70ec10","Type":"ContainerStarted","Data":"9bc480f780b29386fb88e30758504eba313cdada3e0a7f16b5613733cb23c852"}
Sep 16 17:24:55.796502 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.796482    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" event={"ID":"9965d588-9866-4a8c-8e44-ff2596388671","Type":"ContainerStarted","Data":"fbda3e566dbf155d5a01a6f25178120668544b1ef4fe46cb9e03a1652e6da481"}
Sep 16 17:24:55.799466 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.799445    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="a7ca16a1c56e0ca3b11b98aa301215ae26e6729f41efb31340df714c5b3ffad7" exitCode=0
Sep 16 17:24:55.799594 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.799523    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"a7ca16a1c56e0ca3b11b98aa301215ae26e6729f41efb31340df714c5b3ffad7"}
Sep 16 17:24:55.800703 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:55.800680    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerStarted","Data":"eaf0c0591367eba010d94f00da35a19ea0336ba28479ac4abf4afc803ba112c1"}
Sep 16 17:24:56.086101 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:56.086070    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:56.086231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:56.086111    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:56.086287 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:56.086233    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:24:56.086287 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:56.086234    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:24:56.086378 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:56.086309    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:24:58.086288502 +0000 UTC m=+38.078384101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:24:56.086378 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:56.086245    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:24:56.086378 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:56.086363    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:24:58.08635175 +0000 UTC m=+38.078447324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:24:56.816471 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:56.815635    2542 generic.go:356] "Generic (PLEG): container finished" podID="65383bc0-ebcb-4a48-87a1-fcaa1855f17e" containerID="07d304b841c49b173f59ad0cd975a762ecd76d11605a5014942f00372e214439" exitCode=0
Sep 16 17:24:56.816471 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:56.815696    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerDied","Data":"07d304b841c49b173f59ad0cd975a762ecd76d11605a5014942f00372e214439"}
Sep 16 17:24:57.835641 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:57.834914    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" event={"ID":"65383bc0-ebcb-4a48-87a1-fcaa1855f17e","Type":"ContainerStarted","Data":"c199577db467e40b76fb5d5166e13b41492a0313c513aa7ef96069c786d26371"}
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:58.104601    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:58.104654    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:58.104798    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:58.104815    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:58.104899    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:25:02.104878249 +0000 UTC m=+42.096973825 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:58.104899    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:24:58.105036 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:24:58.104964    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:02.104946219 +0000 UTC m=+42.097041792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:24:58.389121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:58.389041    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:24:58.764956 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:58.764840    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:24:59.413554 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:59.413519    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:24:59.573541 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:59.573500    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:24:59.822881 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:24:59.822785    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:00.190396 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:00.190319    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:02.136592 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:02.136553    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:25:02.136592 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:02.136600    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:25:02.137128 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:02.136692    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:02.137128 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:02.136704    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:25:02.137128 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:02.136760    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:25:10.136745925 +0000 UTC m=+50.128841498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:25:02.137128 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:02.136693    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:02.137128 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:02.136818    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:10.136804607 +0000 UTC m=+50.128900180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:04.518207 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.518166    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:04.841323 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.841239    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:04.855062 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.855029    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5d5hd" event={"ID":"60d86da6-1f33-439b-9ee5-a289ed6c00e8","Type":"ContainerStarted","Data":"60b26553f3f06981a6cc5dcd207a8bac2898903a6450098f56503f432b4928df"}
Sep 16 17:25:04.856388 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.856363    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" event={"ID":"4d0a280d-1f96-417a-bbf1-4f083f70ec10","Type":"ContainerStarted","Data":"e2d42acbca9fe38befde1feb3fce81344d27a24a928360667276a0d0bdf68500"}
Sep 16 17:25:04.857789 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.857770    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" event={"ID":"9965d588-9866-4a8c-8e44-ff2596388671","Type":"ContainerStarted","Data":"231c50c348aa72b0affb619206774c346ab6d9f7bd0627464ddffd72dc29904b"}
Sep 16 17:25:04.857916 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.857807    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:25:04.859767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.859748    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:25:04.860080 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.860057    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerStarted","Data":"9403d2ee824c00d0839bac5717783c47c24013c49f0450d0b249651b7be107b8"}
Sep 16 17:25:04.860160 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.860091    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerStarted","Data":"0977aaface954712a263ec10eec05ab4c08abf003c74dc71f90d2ade6687013d"}
Sep 16 17:25:04.860160 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.860108    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerStarted","Data":"b3ee372e1864d9b228d2d399c2eeb5a8be3e6ab7dafa3c18531177e6062a72fc"}
Sep 16 17:25:04.885720 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.885685    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ddjl5" podStartSLOduration=15.072122698 podStartE2EDuration="43.885673326s" totalImagesPullingTime="28.813550628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:24:57.883620821 +0000 UTC m=+37.875716417" watchObservedRunningTime="2026-09-16 17:25:04.885673326 +0000 UTC m=+44.877768970"
Sep 16 17:25:04.885989 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.885968    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5d5hd" podStartSLOduration=2.384904472 podStartE2EDuration="10.885961616s" totalImagesPullingTime="8.501057144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:04.884644755 +0000 UTC m=+44.876740350" watchObservedRunningTime="2026-09-16 17:25:04.885961616 +0000 UTC m=+44.878057203"
Sep 16 17:25:04.926936 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.926895    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" podStartSLOduration=15.426219143 podStartE2EDuration="23.926881977s" totalImagesPullingTime="8.500662834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:04.926594635 +0000 UTC m=+44.918690231" watchObservedRunningTime="2026-09-16 17:25:04.926881977 +0000 UTC m=+44.918977573"
Sep 16 17:25:04.954228 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:04.954188    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" podStartSLOduration=15.417984432 podStartE2EDuration="23.954175299s" totalImagesPullingTime="8.536190867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:04.953537602 +0000 UTC m=+44.945633198" watchObservedRunningTime="2026-09-16 17:25:04.954175299 +0000 UTC m=+44.946270895"
Sep 16 17:25:05.022309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:05.022282    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:09.613924 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:09.613846    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:09.820450 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:09.820415    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:10.015603 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:10.015571    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:10.197890 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:10.197838    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:25:10.198053 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:10.197964    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:25:10.198053 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:10.198008    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:10.198053 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:10.198027    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:25:10.198053 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:10.198053    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:10.198181 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:10.198097    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.198081336 +0000 UTC m=+66.190176909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:25:10.198181 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:10.198111    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:26.198105128 +0000 UTC m=+66.190200702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:11.468421 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:11.468377    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:11.622167 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:11.622128    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:12.122566 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:12.122531    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:12.301790 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:12.301758    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:12.418099 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:12.418060    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:13.415916 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:13.415880    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:13.566758 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:13.566731    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:16.384838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:16.384747    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:16.617227 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:16.617195    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:20.788590 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:20.788561    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mrjcs"
Sep 16 17:25:20.852277 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:20.849526    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" podStartSLOduration=31.329110375 podStartE2EDuration="39.849506596s" totalImagesPullingTime="8.520396221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:05.004901524 +0000 UTC m=+44.996997118" watchObservedRunningTime="2026-09-16 17:25:20.849506596 +0000 UTC m=+60.841602191"
Sep 16 17:25:21.062145 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:21.062058    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:21.177569 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:21.177538    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:21.301152 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:21.301121    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:23.116344 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:23.116314    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:23.330604 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:23.330569    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:25.487042 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:25.487009    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:25.608586 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:25.608557    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:26.187735 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.187706    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:26.218875 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.218838    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:25:26.218950 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.218891    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:25:26.218999 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.218983    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:26.219047 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.219037    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:25:58.21902199 +0000 UTC m=+98.211117564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:25:26.219094 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.218984    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:26.219094 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.219060    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:25:26.219094 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.219083    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:25:58.219076906 +0000 UTC m=+98.211172480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:25:26.320189 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.320161    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:25:26.320325 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.320217    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:25:26.320325 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.320259    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:25:26.323433 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.323416    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 17:25:26.323584 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.323570    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 17:25:26.323995 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.323981    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 17:25:26.330980 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.330961    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 17:25:26.331069 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:26.331054    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:26:30.331018864 +0000 UTC m=+130.323114440 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : secret "metrics-daemon-secret" not found
Sep 16 17:25:26.333949 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.333930    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/34a69a06-bfeb-4602-a1d9-dbd5181b884f-original-pull-secret\") pod \"global-pull-secret-syncer-7knvj\" (UID: \"34a69a06-bfeb-4602-a1d9-dbd5181b884f\") " pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:25:26.336383 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.336369    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 17:25:26.344142 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.344117    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfkxg\" (UniqueName: \"kubernetes.io/projected/42df8adb-ae13-421d-a547-3b5a11d49657-kube-api-access-rfkxg\") pod \"network-check-target-kjfbw\" (UID: \"42df8adb-ae13-421d-a547-3b5a11d49657\") " pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:25:26.531179 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.531112    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2lh24\""
Sep 16 17:25:26.534040 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.534023    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7knvj"
Sep 16 17:25:26.537972 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.537948    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:25:26.821415 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.821348    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kjfbw"]
Sep 16 17:25:26.823839 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.823815    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7knvj"]
Sep 16 17:25:26.824986 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:25:26.824966    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42df8adb_ae13_421d_a547_3b5a11d49657.slice/crio-8a2d03defe93310543589696ade051b02f179dde3b968bd0446c69bee1598794 WatchSource:0}: Error finding container 8a2d03defe93310543589696ade051b02f179dde3b968bd0446c69bee1598794: Status 404 returned error can't find the container with id 8a2d03defe93310543589696ade051b02f179dde3b968bd0446c69bee1598794
Sep 16 17:25:26.826336 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:25:26.826311    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a69a06_bfeb_4602_a1d9_dbd5181b884f.slice/crio-c17f5ffb8f98cef1c8204b4ad16cc97a199403d832389e94d57322374da9477b WatchSource:0}: Error finding container c17f5ffb8f98cef1c8204b4ad16cc97a199403d832389e94d57322374da9477b: Status 404 returned error can't find the container with id c17f5ffb8f98cef1c8204b4ad16cc97a199403d832389e94d57322374da9477b
Sep 16 17:25:26.923032 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.923006    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kjfbw" event={"ID":"42df8adb-ae13-421d-a547-3b5a11d49657","Type":"ContainerStarted","Data":"8a2d03defe93310543589696ade051b02f179dde3b968bd0446c69bee1598794"}
Sep 16 17:25:26.923953 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:26.923927    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7knvj" event={"ID":"34a69a06-bfeb-4602-a1d9-dbd5181b884f","Type":"ContainerStarted","Data":"c17f5ffb8f98cef1c8204b4ad16cc97a199403d832389e94d57322374da9477b"}
Sep 16 17:25:27.838573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:27.838469    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:27.989170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:27.989104    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:31.941950 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:31.941892    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kjfbw" event={"ID":"42df8adb-ae13-421d-a547-3b5a11d49657","Type":"ContainerStarted","Data":"75c849b48a4082514038480d59d6d44a02ebb4eb63c45ed9cf133169ab4e3cdb"}
Sep 16 17:25:31.942385 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:31.942022    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:25:31.943441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:31.943419    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7knvj" event={"ID":"34a69a06-bfeb-4602-a1d9-dbd5181b884f","Type":"ContainerStarted","Data":"dce356151d970b0d978bfa8e40ccee49a926e7d939bcb0eb5fe7f37a031e14ed"}
Sep 16 17:25:31.962346 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:31.962307    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kjfbw" podStartSLOduration=66.24019114 podStartE2EDuration="1m10.962260045s" totalImagesPullingTime="4.722068905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:31.96117641 +0000 UTC m=+71.953271995" watchObservedRunningTime="2026-09-16 17:25:31.962260045 +0000 UTC m=+71.954355641"
Sep 16 17:25:31.979773 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:31.979733    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7knvj" podStartSLOduration=66.253678714 podStartE2EDuration="1m10.979719873s" totalImagesPullingTime="4.726041159s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:25:31.978823925 +0000 UTC m=+71.970919522" watchObservedRunningTime="2026-09-16 17:25:31.979719873 +0000 UTC m=+71.971815471"
Sep 16 17:25:36.429745 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:36.429705    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:36.525243 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:36.525213    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:46.260220 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:46.260096    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:47.456687 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:47.456646    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:47.557712 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:47.557675    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:25:58.239406 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:58.239366    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:25:58.239406 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:25:58.239408    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:25:58.239824 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:58.239511    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:25:58.239824 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:58.239524    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:25:58.239824 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:58.239578    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:27:02.239564494 +0000 UTC m=+162.231660081 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:25:58.239824 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:58.239509    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:25:58.239824 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:25:58.239656    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:02.239643517 +0000 UTC m=+162.231739091 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:26:02.949211 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:02.949182    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kjfbw"
Sep 16 17:26:30.376080 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:30.376039    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:26:30.376537 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:30.376193    2542 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 16 17:26:30.376537 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:30.376262    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs podName:f313bc9f-a278-4e04-aaef-e2e9f30c18db nodeName:}" failed. No retries permitted until 2026-09-16 17:28:32.376241279 +0000 UTC m=+252.368336857 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs") pod "network-metrics-daemon-s76cv" (UID: "f313bc9f-a278-4e04-aaef-e2e9f30c18db") : secret "metrics-daemon-secret" not found
Sep 16 17:26:48.706196 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:48.706164    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:50.252564 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:50.252515    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:51.574349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:51.574317    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:51.675993 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:51.675964    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.173938 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.173903    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pmhdq"]
Sep 16 17:26:53.177010 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.176987    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.179403 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.179359    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 17:26:53.179575 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.179561    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 17:26:53.180461 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.180443    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mfnhp\""
Sep 16 17:26:53.180573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.180460    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 17:26:53.180573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.180513    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 17:26:53.190765 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.190742    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmhdq"]
Sep 16 17:26:53.213573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.213546    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.229611 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.229587    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1713059b-bf0c-46fd-9b41-2a70816d3645-crio-socket\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.229713 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.229616    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.229713 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.229652    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1713059b-bf0c-46fd-9b41-2a70816d3645-data-volume\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.229801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.229732    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.229801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.229749    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbpf9\" (UniqueName: \"kubernetes.io/projected/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-api-access-jbpf9\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.286905 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.286867    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.331035 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.330997    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1713059b-bf0c-46fd-9b41-2a70816d3645-crio-socket\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331157 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331045    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331157 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331085    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1713059b-bf0c-46fd-9b41-2a70816d3645-data-volume\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331157 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331122    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1713059b-bf0c-46fd-9b41-2a70816d3645-crio-socket\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331131    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.331196    2542 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331200    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpf9\" (UniqueName: \"kubernetes.io/projected/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-api-access-jbpf9\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.331231    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.331274    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:53.83125442 +0000 UTC m=+153.823350011 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : configmap "kube-rbac-proxy" not found
Sep 16 17:26:53.331308 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.331303    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:53.831283373 +0000 UTC m=+153.823378961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:53.331510 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.331397    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1713059b-bf0c-46fd-9b41-2a70816d3645-data-volume\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.342212 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.342187    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbpf9\" (UniqueName: \"kubernetes.io/projected/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-api-access-jbpf9\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.393679 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.393648    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.491166 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.491080    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.559499 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.559470    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.834570 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.834481    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.834570 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.834534    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:53.834765 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.834602    2542 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:26:53.834765 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.834626    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:53.834765 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.834671    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:54.834654622 +0000 UTC m=+154.826750213 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : configmap "kube-rbac-proxy" not found
Sep 16 17:26:53.834765 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:53.834689    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:54.834682887 +0000 UTC m=+154.826778461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:53.917755 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.917718    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:53.991846 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:53.991789    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:54.842194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:54.842158    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:54.842561 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:54.842313    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:54.842561 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:54.842314    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:54.842561 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:54.842344    2542 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:26:54.842561 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:54.842392    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:56.842377162 +0000 UTC m=+156.834472737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:54.842561 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:54.842406    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:26:56.842399416 +0000 UTC m=+156.834494990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : configmap "kube-rbac-proxy" not found
Sep 16 17:26:55.115355 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:55.115287    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-czk72_49da429c-403a-46cf-a214-c584c906f98f/dns-node-resolver/0.log"
Sep 16 17:26:55.557553 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:55.557516    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:55.657656 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:55.657619    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:55.760384 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:55.760313    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:55.913665 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:55.913640    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-28mvz_74e282fd-2c09-4e91-af47-b93ffce7077c/node-ca/0.log"
Sep 16 17:26:56.507160 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:56.507122    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:26:56.857814 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:56.857719    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:56.858021 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:56.857904    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:26:56.858021 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:56.857916    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:56.858021 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:56.857990    2542 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:26:56.858021 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:56.857992    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:00.857973237 +0000 UTC m=+160.850068811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:26:56.858240 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:56.858039    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:00.858027293 +0000 UTC m=+160.850122867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : configmap "kube-rbac-proxy" not found
Sep 16 17:26:56.915055 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:56.915030    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5d5hd_60d86da6-1f33-439b-9ee5-a289ed6c00e8/serve-healthcheck-canary/0.log"
Sep 16 17:26:57.325261 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:57.325225    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-84c4f75687-wklww" podUID="9da11ef8-21a0-445f-9090-7f45826fd04d"
Sep 16 17:26:57.377222 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:57.377175    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rkb45" podUID="d92892f1-d976-4c50-ab9d-0398376fb6a1"
Sep 16 17:26:58.193385 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:58.193356    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:26:58.193819 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:26:58.193356    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:26:58.640229 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:26:58.640153    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-s76cv" podUID="f313bc9f-a278-4e04-aaef-e2e9f30c18db"
Sep 16 17:27:00.085725 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.085690    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.186555 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.186522    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.287407 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.287362    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.389293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.389171    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.487679 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.487651    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.593406 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.593378    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.795135 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.795106    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:00.885554 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.885522    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:00.885652 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:00.885592    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:00.885696 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:00.885661    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:00.885696 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:00.885670    2542 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 17:27:00.885774 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:00.885717    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:08.885701499 +0000 UTC m=+168.877797077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:00.885774 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:00.885731    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:08.885725329 +0000 UTC m=+168.877820902 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : configmap "kube-rbac-proxy" not found
Sep 16 17:27:01.785723 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:01.785688    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:02.192242 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.192208    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:02.297219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.297192    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:27:02.297367 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.297226    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") pod \"image-registry-84c4f75687-wklww\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") " pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:27:02.297367 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:02.297334    2542 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 17:27:02.297367 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:02.297359    2542 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 17:27:02.297463 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:02.297370    2542 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-84c4f75687-wklww: secret "image-registry-tls" not found
Sep 16 17:27:02.297463 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:02.297391    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls podName:d92892f1-d976-4c50-ab9d-0398376fb6a1 nodeName:}" failed. No retries permitted until 2026-09-16 17:29:04.297376028 +0000 UTC m=+284.289471603 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls") pod "dns-default-rkb45" (UID: "d92892f1-d976-4c50-ab9d-0398376fb6a1") : secret "dns-default-metrics-tls" not found
Sep 16 17:27:02.297463 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:02.297410    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls podName:9da11ef8-21a0-445f-9090-7f45826fd04d nodeName:}" failed. No retries permitted until 2026-09-16 17:29:04.297399757 +0000 UTC m=+284.289495331 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls") pod "image-registry-84c4f75687-wklww" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d") : secret "image-registry-tls" not found
Sep 16 17:27:02.370194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.370166    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:02.583262 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.583180    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:02.777523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:02.777493    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:03.619293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:03.619257    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:03.813152 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:03.813119    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.002010 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.001980    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.182927 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.182872    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.387170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.387088    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.466812 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.466782    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.597607 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.597584    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:04.643943 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.643914    2542 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" podUID="4d0a280d-1f96-417a-bbf1-4f083f70ec10" containerName="addon-agent" probeResult="failure" output="Get \"http://10.132.0.7:8000/healthz\": dial tcp 10.132.0.7:8000: connect: connection refused"
Sep 16 17:27:04.651424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.651402    2542 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" podUID="9965d588-9866-4a8c-8e44-ff2596388671" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/healthz\": dial tcp 10.132.0.9:8000: connect: connection refused"
Sep 16 17:27:04.858167 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.858129    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" podUID="9965d588-9866-4a8c-8e44-ff2596388671" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused"
Sep 16 17:27:04.988398 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:04.988291    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:05.213898 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.213840    2542 generic.go:356] "Generic (PLEG): container finished" podID="4d0a280d-1f96-417a-bbf1-4f083f70ec10" containerID="e2d42acbca9fe38befde1feb3fce81344d27a24a928360667276a0d0bdf68500" exitCode=255
Sep 16 17:27:05.214059 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.213916    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" event={"ID":"4d0a280d-1f96-417a-bbf1-4f083f70ec10","Type":"ContainerDied","Data":"e2d42acbca9fe38befde1feb3fce81344d27a24a928360667276a0d0bdf68500"}
Sep 16 17:27:05.214266 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.214240    2542 scope.go:117] "RemoveContainer" containerID="e2d42acbca9fe38befde1feb3fce81344d27a24a928360667276a0d0bdf68500"
Sep 16 17:27:05.215390 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.215369    2542 generic.go:356] "Generic (PLEG): container finished" podID="9965d588-9866-4a8c-8e44-ff2596388671" containerID="231c50c348aa72b0affb619206774c346ab6d9f7bd0627464ddffd72dc29904b" exitCode=1
Sep 16 17:27:05.215475 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.215423    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" event={"ID":"9965d588-9866-4a8c-8e44-ff2596388671","Type":"ContainerDied","Data":"231c50c348aa72b0affb619206774c346ab6d9f7bd0627464ddffd72dc29904b"}
Sep 16 17:27:05.215684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.215671    2542 scope.go:117] "RemoveContainer" containerID="231c50c348aa72b0affb619206774c346ab6d9f7bd0627464ddffd72dc29904b"
Sep 16 17:27:05.590316 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:05.590240    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:06.037353 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.037324    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:06.219824 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.219794    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-95b756b44-2jkn6" event={"ID":"4d0a280d-1f96-417a-bbf1-4f083f70ec10","Type":"ContainerStarted","Data":"5958bafec8131be5124302481fa7f14d8bcda749ee12fd16d47ab2647420b7f5"}
Sep 16 17:27:06.221645 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.221623    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx" event={"ID":"9965d588-9866-4a8c-8e44-ff2596388671","Type":"ContainerStarted","Data":"0208f663b9f7621ce82c77dc1443e58eb7607db171537334b5df4545a84adfb1"}
Sep 16 17:27:06.221894 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.221872    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:27:06.222711 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.222694    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74dfcfb986-rrhlx"
Sep 16 17:27:06.457005 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.456970    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:06.523413 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.523388    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:06.631923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:06.631890    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:07.031865 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:07.031828    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:07.792796 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:07.792769    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:07.995623 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:07.995584    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:08.083346 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.083263    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:08.193174 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.193145    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:08.388993 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.388921    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:08.950229 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.950194    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:08.950594 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.950245    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:08.950594 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:08.950334    2542 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:08.950594 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:08.950385    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls podName:1713059b-bf0c-46fd-9b41-2a70816d3645 nodeName:}" failed. No retries permitted until 2026-09-16 17:27:24.950371899 +0000 UTC m=+184.942467473 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pmhdq" (UID: "1713059b-bf0c-46fd-9b41-2a70816d3645") : secret "insights-runtime-extractor-tls" not found
Sep 16 17:27:08.950724 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:08.950696    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1713059b-bf0c-46fd-9b41-2a70816d3645-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:09.576046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:09.576016    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:09.773684 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:09.773645    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:09.988123 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:09.988090    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:10.188706 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:10.188668    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:10.385629 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:10.385554    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:10.791467 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:10.791432    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:13.617722 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:13.617638    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:27:14.818077 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:14.818039    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:14.919881 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:14.919831    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:15.126588 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:15.126515    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:15.514263 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:15.514232    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:15.572702 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:15.572681    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:15.894764 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:15.894738    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:15.966027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:15.965992    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:16.307111 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:16.307037    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:16.731697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:16.731660    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:16.828746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:16.828704    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:17.390726 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:17.390691    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:17.573246 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:17.573216    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:17.814236 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:17.814194    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:17.919013 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:17.918980    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:18.061157 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:18.061124    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:18.236501 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:18.236470    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:19.791794 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:19.791768    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:19.858165 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:19.858134    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:22.889977 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:22.889940    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:22.976690 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:22.976661    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:23.081741 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:23.081709    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:23.462344 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:23.462311    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:24.974972 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:24.974937    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:24.977150 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:24.977129    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1713059b-bf0c-46fd-9b41-2a70816d3645-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pmhdq\" (UID: \"1713059b-bf0c-46fd-9b41-2a70816d3645\") " pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:24.989499 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:24.989475    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mfnhp\""
Sep 16 17:27:24.995743 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:24.995728    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pmhdq"
Sep 16 17:27:25.265768 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:25.265747    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pmhdq"]
Sep 16 17:27:25.267904 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:27:25.267877    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1713059b_bf0c_46fd_9b41_2a70816d3645.slice/crio-c854a4c3d3636d6b437089f81593c27c7a3f2c50681f2ed85267b2242c6d2dd2 WatchSource:0}: Error finding container c854a4c3d3636d6b437089f81593c27c7a3f2c50681f2ed85267b2242c6d2dd2: Status 404 returned error can't find the container with id c854a4c3d3636d6b437089f81593c27c7a3f2c50681f2ed85267b2242c6d2dd2
Sep 16 17:27:25.285356 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:25.285331    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmhdq" event={"ID":"1713059b-bf0c-46fd-9b41-2a70816d3645","Type":"ContainerStarted","Data":"c854a4c3d3636d6b437089f81593c27c7a3f2c50681f2ed85267b2242c6d2dd2"}
Sep 16 17:27:26.290677 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:26.290641    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmhdq" event={"ID":"1713059b-bf0c-46fd-9b41-2a70816d3645","Type":"ContainerStarted","Data":"55e9c1601964d30db08a36538b64fb8e66d9fdab4b507ed2d45b9c437931d0aa"}
Sep 16 17:27:26.291143 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:26.290698    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmhdq" event={"ID":"1713059b-bf0c-46fd-9b41-2a70816d3645","Type":"ContainerStarted","Data":"7d77b6fb5576dc4def1c42be5b46de200a7507d527bffc4d8ccf0f0dfe5190d3"}
Sep 16 17:27:27.141404 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:27.141371    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:27.279528 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:27.279497    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:27.298904 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:27.295821    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pmhdq" event={"ID":"1713059b-bf0c-46fd-9b41-2a70816d3645","Type":"ContainerStarted","Data":"19f54fa1ffe544afb88a2ef9d2baa2787948a8b22dc77e4bc97724a59fbccf0f"}
Sep 16 17:27:27.319800 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:27.319714    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pmhdq" podStartSLOduration=32.626183592 podStartE2EDuration="34.319700928s" totalImagesPullingTime="1.693517336s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:26:53 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:27.318264206 +0000 UTC m=+187.310359802" watchObservedRunningTime="2026-09-16 17:27:27.319700928 +0000 UTC m=+187.311796523"
Sep 16 17:27:27.688514 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:27.688483    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:30.126743 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:30.126705    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:30.390644 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:30.390555    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:30.506252 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:30.506220    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:32.344744 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.344710    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:32.558237 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.558205    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:32.709099 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.709069    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:32.752780 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.752753    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2dd2c"]
Sep 16 17:27:32.756536 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.756511    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.760573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.760546    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 17:27:32.760573 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.760548    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 17:27:32.760832 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.760817    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 17:27:32.761523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.761507    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rg679\""
Sep 16 17:27:32.761621 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.761523    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 17:27:32.761621 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.761536    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 17:27:32.761621 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.761550    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 17:27:32.831863 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831833    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.831983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831878    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-root\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.831983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831919    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-tls\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.831983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831936    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.831983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831952    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-wtmp\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.831983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.831969    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-metrics-client-ca\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.832168 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.832013    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-textfile\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.832168 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.832037    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-sys\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.832168 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.832058    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92764\" (UniqueName: \"kubernetes.io/projected/c0b11b96-936e-48e4-8678-02aa0e249d47-kube-api-access-92764\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.896834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.896801    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:32.933179 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933157    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92764\" (UniqueName: \"kubernetes.io/projected/c0b11b96-936e-48e4-8678-02aa0e249d47-kube-api-access-92764\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933323 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933201    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933418 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933402    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-root\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933477 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933447    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-tls\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933477 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933465    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933571 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933482    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-wtmp\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933571 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933510    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-root\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933571 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933513    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-metrics-client-ca\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933571 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933556    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-textfile\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933725 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933587    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-sys\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933725 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933611    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-wtmp\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933725 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933659    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c0b11b96-936e-48e4-8678-02aa0e249d47-sys\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933819 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933750    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-accelerators-collector-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.933934 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933914    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-textfile\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.934003 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.933992    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c0b11b96-936e-48e4-8678-02aa0e249d47-metrics-client-ca\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.935845 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.935823    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.935927 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.935912    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c0b11b96-936e-48e4-8678-02aa0e249d47-node-exporter-tls\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:32.943163 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:32.943145    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92764\" (UniqueName: \"kubernetes.io/projected/c0b11b96-936e-48e4-8678-02aa0e249d47-kube-api-access-92764\") pod \"node-exporter-2dd2c\" (UID: \"c0b11b96-936e-48e4-8678-02aa0e249d47\") " pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:33.065772 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.065699    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2dd2c"
Sep 16 17:27:33.075538 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:27:33.075515    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b11b96_936e_48e4_8678_02aa0e249d47.slice/crio-64ee3bd83582908447676e83bef30ee79e1433089346f0463131d1dfddd90942 WatchSource:0}: Error finding container 64ee3bd83582908447676e83bef30ee79e1433089346f0463131d1dfddd90942: Status 404 returned error can't find the container with id 64ee3bd83582908447676e83bef30ee79e1433089346f0463131d1dfddd90942
Sep 16 17:27:33.108424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.108401    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:33.316136 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.316052    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dd2c" event={"ID":"c0b11b96-936e-48e4-8678-02aa0e249d47","Type":"ContainerStarted","Data":"64ee3bd83582908447676e83bef30ee79e1433089346f0463131d1dfddd90942"}
Sep 16 17:27:33.370078 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.370046    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:33.546190 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.546148    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:33.656891 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.656839    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:33.768742 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:33.768709    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:34.088870 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:34.088781    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:34.321298 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:34.321271    2542 generic.go:356] "Generic (PLEG): container finished" podID="c0b11b96-936e-48e4-8678-02aa0e249d47" containerID="11353ed9a8e0f2dc660b5ff2d47e2f8eb46b6306d5fa52fb4f8b3fbf68357ef0" exitCode=0
Sep 16 17:27:34.321425 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:34.321332    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dd2c" event={"ID":"c0b11b96-936e-48e4-8678-02aa0e249d47","Type":"ContainerDied","Data":"11353ed9a8e0f2dc660b5ff2d47e2f8eb46b6306d5fa52fb4f8b3fbf68357ef0"}
Sep 16 17:27:34.670406 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:34.670369    2542 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" podUID="417fecd4-200c-4610-bc72-a17ea41bb2af" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 17:27:35.327078 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:35.327045    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dd2c" event={"ID":"c0b11b96-936e-48e4-8678-02aa0e249d47","Type":"ContainerStarted","Data":"a6f4e51b0d5118ee1f219cf999efc421e4fdbafb01c0fe082b794ef66bb03539"}
Sep 16 17:27:35.327237 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:35.327093    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2dd2c" event={"ID":"c0b11b96-936e-48e4-8678-02aa0e249d47","Type":"ContainerStarted","Data":"643f31b64ba26110cbad2b812abac57da7c242844d2376bcf8e733f26f406448"}
Sep 16 17:27:35.352673 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:35.352626    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2dd2c" podStartSLOduration=2.208365808 podStartE2EDuration="3.352608553s" totalImagesPullingTime="1.083619636s" totalInitContainerRuntime="60.623109ms" isStatefulPod=true podCreationTimestamp="2026-09-16 17:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:27:35.352282234 +0000 UTC m=+195.344377832" watchObservedRunningTime="2026-09-16 17:27:35.352608553 +0000 UTC m=+195.344704149"
Sep 16 17:27:35.804990 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:35.804950    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:35.942324 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:35.942289    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:36.073313 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:36.073214    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:36.136400 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:36.136372    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:37.144291 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:37.144250    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:37.237739 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:37.237709    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:37.357716 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:37.357680    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:37.459630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:37.459551    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:37.602986 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:37.602955    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:39.246564 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:39.246526    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:39.705862 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:39.705826    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c4f75687-wklww"]
Sep 16 17:27:39.706080 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:27:39.706047    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-84c4f75687-wklww" podUID="9da11ef8-21a0-445f-9090-7f45826fd04d"
Sep 16 17:27:39.864667 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:39.864633    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:39.990317 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:39.990229    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:40.103509 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.103481    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:40.341945 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.341846    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:27:40.348088 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.348068    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:27:40.494029 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.493997    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494029 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494036    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494064    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494085    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494109    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6v2q\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494130    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494151    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token\") pod \"9da11ef8-21a0-445f-9090-7f45826fd04d\" (UID: \"9da11ef8-21a0-445f-9090-7f45826fd04d\") "
Sep 16 17:27:40.494523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494374    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:27:40.494523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494451    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:27:40.494523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.494503    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:27:40.496488 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.496452    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q" (OuterVolumeSpecName: "kube-api-access-w6v2q") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "kube-api-access-w6v2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:27:40.496488 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.496483    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:27:40.496682 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.496661    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:27:40.496767 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.496749    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9da11ef8-21a0-445f-9090-7f45826fd04d" (UID: "9da11ef8-21a0-445f-9090-7f45826fd04d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595254    2542 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-image-registry-private-configuration\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595280    2542 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-trusted-ca\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595292    2542 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9da11ef8-21a0-445f-9090-7f45826fd04d-installation-pull-secrets\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595302    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6v2q\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-kube-api-access-w6v2q\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595310    2542 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-certificates\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595318    2542 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-bound-sa-token\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:40.595328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:40.595326    2542 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9da11ef8-21a0-445f-9090-7f45826fd04d-ca-trust-extracted\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:41.344510 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:41.344479    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84c4f75687-wklww"
Sep 16 17:27:41.381976 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:41.381947    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-84c4f75687-wklww"]
Sep 16 17:27:41.387757 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:41.387722    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-84c4f75687-wklww"]
Sep 16 17:27:41.502006 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:41.501974    2542 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9da11ef8-21a0-445f-9090-7f45826fd04d-registry-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:27:42.131184 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:42.131146    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:42.193779 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:42.193748    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:42.622038 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:42.622006    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9da11ef8-21a0-445f-9090-7f45826fd04d" path="/var/lib/kubelet/pods/9da11ef8-21a0-445f-9090-7f45826fd04d/volumes"
Sep 16 17:27:42.853087 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:42.853054    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:44.671085 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:44.671052    2542 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" podUID="417fecd4-200c-4610-bc72-a17ea41bb2af" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 17:27:45.077790 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:45.077711    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:45.209083 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:45.209041    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:47.508729 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:47.508689    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:47.616736 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:47.616708    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:49.113512 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:49.113481    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:49.236478 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:49.236430    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:50.634948 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:50.634914    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:50.716977 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:50.716942    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:52.139754 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:52.139726    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:52.212710 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:52.212679    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:54.669997 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:54.669957    2542 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" podUID="417fecd4-200c-4610-bc72-a17ea41bb2af" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 16 17:27:54.670354 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:54.670032    2542 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk"
Sep 16 17:27:54.670506 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:54.670477    2542 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9403d2ee824c00d0839bac5717783c47c24013c49f0450d0b249651b7be107b8"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 16 17:27:54.670543 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:54.670527    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" podUID="417fecd4-200c-4610-bc72-a17ea41bb2af" containerName="service-proxy" containerID="cri-o://9403d2ee824c00d0839bac5717783c47c24013c49f0450d0b249651b7be107b8" gracePeriod=30
Sep 16 17:27:55.388344 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:55.388313    2542 generic.go:356] "Generic (PLEG): container finished" podID="417fecd4-200c-4610-bc72-a17ea41bb2af" containerID="9403d2ee824c00d0839bac5717783c47c24013c49f0450d0b249651b7be107b8" exitCode=2
Sep 16 17:27:55.388508 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:55.388368    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerDied","Data":"9403d2ee824c00d0839bac5717783c47c24013c49f0450d0b249651b7be107b8"}
Sep 16 17:27:55.388508 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:55.388411    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6c6f77989d-pg6gk" event={"ID":"417fecd4-200c-4610-bc72-a17ea41bb2af","Type":"ContainerStarted","Data":"f50b624aa8378ce76b4e7be3419cfc3de94c542efd23e7aa08923acb8cbdec0c"}
Sep 16 17:27:59.688152 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:59.688117    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:27:59.788111 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:27:59.788082    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:04.017271 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:04.017240    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:04.112974 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:04.112944    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:04.794272 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:04.794236    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:04.875197 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:04.875163    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:04.949219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:04.949194    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-czk72_49da429c-403a-46cf-a214-c584c906f98f/dns-node-resolver/0.log"
Sep 16 17:28:05.615321 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:05.615285    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:05.714299 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:05.714267    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:05.758756 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:05.758734    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5d5hd_60d86da6-1f33-439b-9ee5-a289ed6c00e8/serve-healthcheck-canary/0.log"
Sep 16 17:28:05.803562 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:05.803535    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:19.783326 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:19.783294    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:19.850141 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:19.850104    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:32.455478 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:32.455444    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:28:32.457721 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:32.457697    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f313bc9f-a278-4e04-aaef-e2e9f30c18db-metrics-certs\") pod \"network-metrics-daemon-s76cv\" (UID: \"f313bc9f-a278-4e04-aaef-e2e9f30c18db\") " pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:28:32.521266 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:32.521241    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q452j\""
Sep 16 17:28:32.529443 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:32.529428    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s76cv"
Sep 16 17:28:32.797495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:32.797431    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s76cv"]
Sep 16 17:28:32.800407 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:28:32.800379    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf313bc9f_a278_4e04_aaef_e2e9f30c18db.slice/crio-f4b9bae5b6ccb7383bf7cdcd6d95279b3670106af96125015bd87b46f5873ccb WatchSource:0}: Error finding container f4b9bae5b6ccb7383bf7cdcd6d95279b3670106af96125015bd87b46f5873ccb: Status 404 returned error can't find the container with id f4b9bae5b6ccb7383bf7cdcd6d95279b3670106af96125015bd87b46f5873ccb
Sep 16 17:28:33.510270 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:33.510238    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s76cv" event={"ID":"f313bc9f-a278-4e04-aaef-e2e9f30c18db","Type":"ContainerStarted","Data":"f4b9bae5b6ccb7383bf7cdcd6d95279b3670106af96125015bd87b46f5873ccb"}
Sep 16 17:28:34.515508 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:34.515473    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s76cv" event={"ID":"f313bc9f-a278-4e04-aaef-e2e9f30c18db","Type":"ContainerStarted","Data":"c6c9401f47ca21a0c97cd630a60f98ce7d4a8e296b0ed89f6da99a4818a75a0d"}
Sep 16 17:28:34.515938 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:34.515527    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s76cv" event={"ID":"f313bc9f-a278-4e04-aaef-e2e9f30c18db","Type":"ContainerStarted","Data":"4b705aad975ac566f44010f0657d04dd885f91b6647a360b9b05436cc4f75d5d"}
Sep 16 17:28:34.534110 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:34.534059    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s76cv" podStartSLOduration=252.5797781 podStartE2EDuration="4m13.534043984s" totalImagesPullingTime="954.265884ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:28:34.532391754 +0000 UTC m=+254.524487354" watchObservedRunningTime="2026-09-16 17:28:34.534043984 +0000 UTC m=+254.526139578"
Sep 16 17:28:34.638563 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:34.638532    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:34.714288 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:34.714253    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:51.092292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:51.092220    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:51.157492 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:51.157457    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:52.073934 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:52.073894    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:28:52.771836 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:28:52.771802    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:00.387061 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:00.387028    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:00.483873 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:00.483823    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:01.194673 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:29:01.194637    2542 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-rkb45" podUID="d92892f1-d976-4c50-ab9d-0398376fb6a1"
Sep 16 17:29:01.600419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:01.600342    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:02.259094 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:02.259050    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:02.360351 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:02.360323    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:04.365242 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:04.365211    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:04.367524 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:04.367500    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d92892f1-d976-4c50-ab9d-0398376fb6a1-metrics-tls\") pod \"dns-default-rkb45\" (UID: \"d92892f1-d976-4c50-ab9d-0398376fb6a1\") " pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:04.603746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:04.603715    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v8tmk\""
Sep 16 17:29:04.611481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:04.611459    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:04.886586 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:04.886514    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rkb45"]
Sep 16 17:29:04.889164 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:29:04.889140    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd92892f1_d976_4c50_ab9d_0398376fb6a1.slice/crio-8437b664549f82ac11b1f3abbba6338063da973de3c7d59efcf56a7f06f5e633 WatchSource:0}: Error finding container 8437b664549f82ac11b1f3abbba6338063da973de3c7d59efcf56a7f06f5e633: Status 404 returned error can't find the container with id 8437b664549f82ac11b1f3abbba6338063da973de3c7d59efcf56a7f06f5e633
Sep 16 17:29:05.613747 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:05.613707    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rkb45" event={"ID":"d92892f1-d976-4c50-ab9d-0398376fb6a1","Type":"ContainerStarted","Data":"8437b664549f82ac11b1f3abbba6338063da973de3c7d59efcf56a7f06f5e633"}
Sep 16 17:29:06.626828 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:06.626796    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rkb45" event={"ID":"d92892f1-d976-4c50-ab9d-0398376fb6a1","Type":"ContainerStarted","Data":"a1829b0da7bfcd5f9bf52e0d347b62c6ee2ce9a79f580d53e7af05ee437775fa"}
Sep 16 17:29:07.631162 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:07.631123    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rkb45" event={"ID":"d92892f1-d976-4c50-ab9d-0398376fb6a1","Type":"ContainerStarted","Data":"65dace01a2d4b4330b36435efc89ea0c90635e4cf17e14e836d7f317ef76e7cd"}
Sep 16 17:29:07.631522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:07.631212    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:07.656513 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:07.656469    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rkb45" podStartSLOduration=252.04890208 podStartE2EDuration="4m13.656453862s" totalImagesPullingTime="1.607551782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:24:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:29:07.654487347 +0000 UTC m=+287.646582942" watchObservedRunningTime="2026-09-16 17:29:07.656453862 +0000 UTC m=+287.648549457"
Sep 16 17:29:10.721446 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:10.721413    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:11.376970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:11.376936    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:11.477215 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:11.477178    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:11.548845 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:11.548818    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:17.636968 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:17.636938    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rkb45"
Sep 16 17:29:20.418477 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:20.418448    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:29:20.421587 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:20.421566    2542 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 17:29:20.422980 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:20.422961    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:29:26.841108 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:26.841078    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:26.942482 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:26.942446    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:33.577046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:33.577004    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:29:33.675988 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:29:33.675947    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:07.289312 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:07.289272    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:07.357074 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:07.357032    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:14.922026 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:14.921936    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:14.989313 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:14.989285    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:15.051501 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:15.051468    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:15.253268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:15.253199    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:15.831524 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:15.831492    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:15.898816 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:15.898784    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:19.296309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:19.296279    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:19.421716 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:19.421682    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:25.649791 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:25.649761    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:25.751463 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:25.751432    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:25.827194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:25.827152    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:34.859661 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:34.859627    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:30:34.943805 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:30:34.943770    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:29.609074 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:29.609035    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:31.004221 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:31.004183    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:40.247130 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:40.247092    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:40.347073 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:40.347028    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:40.446217 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:40.446188    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:44.741604 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:44.741563    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:44.841788 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:44.841756    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:58.200453 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:58.200414    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:58.358669 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:58.358637    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:32:58.515744 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:32:58.515665    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:11.526835 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:11.526800    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:13.290622 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:13.290549    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:13.355045 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:13.355018    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:14.154565 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:14.154535    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:22.321332 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:22.321290    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:22.523302 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:22.523264    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:29.051082 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:29.051040    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:29.679791 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:29.679758    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:30.044241 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.044158    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:30.336676 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.336586    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:30.384956 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.384927    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"]
Sep 16 17:33:30.387984 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.387964    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.404140 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.404116    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Sep 16 17:33:30.404240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.404122    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 16 17:33:30.404466 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.404451    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 16 17:33:30.405142 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.405127    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-q9dmd\""
Sep 16 17:33:30.405667 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.405653    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Sep 16 17:33:30.405707 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.405653    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 16 17:33:30.452127 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.452103    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"]
Sep 16 17:33:30.479441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.479417    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.479540 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.479451    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.479540 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.479480    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9lgm\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-kube-api-access-t9lgm\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.580774 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.580740    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.580774 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.580781    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.580990 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.580806    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9lgm\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-kube-api-access-t9lgm\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.580990 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:33:30.580956    2542 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found
Sep 16 17:33:30.580990 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:33:30.580983    2542 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq: secret "keda-metrics-apiserver-certs" not found
Sep 16 17:33:30.581096 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:33:30.581051    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates podName:ceabd5a2-16d8-43d4-9437-5a72d325b6e7 nodeName:}" failed. No retries permitted until 2026-09-16 17:33:31.081030434 +0000 UTC m=+551.073126008 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates") pod "keda-metrics-apiserver-77857b7b98-2zftq" (UID: "ceabd5a2-16d8-43d4-9437-5a72d325b6e7") : secret "keda-metrics-apiserver-certs" not found
Sep 16 17:33:30.581168 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.581148    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.598141 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.598076    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9lgm\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-kube-api-access-t9lgm\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:30.738169 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.738139    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:30.767559 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.767533    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-5dw67"]
Sep 16 17:33:30.770486 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.770472    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.782275 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.782244    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-certificates\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.782386 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.782298    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bk8c\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-kube-api-access-7bk8c\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.783461 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.783445    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 16 17:33:30.882738 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.882683    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-certificates\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.882738 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.882719    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bk8c\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-kube-api-access-7bk8c\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.884997 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.884981    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-certificates\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:30.912339 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.912312    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-5dw67"]
Sep 16 17:33:30.915697 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:30.915675    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bk8c\" (UniqueName: \"kubernetes.io/projected/43dae30a-a63d-491c-aeea-28ba9f65c85b-kube-api-access-7bk8c\") pod \"keda-admission-6c4d864b86-5dw67\" (UID: \"43dae30a-a63d-491c-aeea-28ba9f65c85b\") " pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:31.080064 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.080031    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:31.084163 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.084146    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:31.086649 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.086629    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ceabd5a2-16d8-43d4-9437-5a72d325b6e7-certificates\") pod \"keda-metrics-apiserver-77857b7b98-2zftq\" (UID: \"ceabd5a2-16d8-43d4-9437-5a72d325b6e7\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:31.102365 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.102337    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:31.297134 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.297101    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:31.329691 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.329615    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:31.364230 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.364206    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-5dw67"]
Sep 16 17:33:31.368798 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:33:31.368774    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dae30a_a63d_491c_aeea_28ba9f65c85b.slice/crio-0741547f5f749ccb54e280e7bd19897d9590fc0b3f1f017b9fe49aecc5d1d363 WatchSource:0}: Error finding container 0741547f5f749ccb54e280e7bd19897d9590fc0b3f1f017b9fe49aecc5d1d363: Status 404 returned error can't find the container with id 0741547f5f749ccb54e280e7bd19897d9590fc0b3f1f017b9fe49aecc5d1d363
Sep 16 17:33:31.370495 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.370478    2542 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:33:31.476662 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.476632    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-5dw67" event={"ID":"43dae30a-a63d-491c-aeea-28ba9f65c85b","Type":"ContainerStarted","Data":"0741547f5f749ccb54e280e7bd19897d9590fc0b3f1f017b9fe49aecc5d1d363"}
Sep 16 17:33:31.523321 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.523294    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:31.590517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.590492    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"]
Sep 16 17:33:31.590641 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:33:31.590628    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceabd5a2_16d8_43d4_9437_5a72d325b6e7.slice/crio-3adf4c968428924501f05e29620cb512efd59570840d000f7831d661935bd0cd WatchSource:0}: Error finding container 3adf4c968428924501f05e29620cb512efd59570840d000f7831d661935bd0cd: Status 404 returned error can't find the container with id 3adf4c968428924501f05e29620cb512efd59570840d000f7831d661935bd0cd
Sep 16 17:33:31.834565 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:31.834535    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:32.228671 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:32.228639    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:32.411601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:32.411567    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:33:32.481923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:32.481839    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq" event={"ID":"ceabd5a2-16d8-43d4-9437-5a72d325b6e7","Type":"ContainerStarted","Data":"3adf4c968428924501f05e29620cb512efd59570840d000f7831d661935bd0cd"}
Sep 16 17:33:35.494889 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.494775    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq" event={"ID":"ceabd5a2-16d8-43d4-9437-5a72d325b6e7","Type":"ContainerStarted","Data":"c47901ae0122d81fa64a64984f1f4efc62d8eac4af3031b1179e27df187e0066"}
Sep 16 17:33:35.495320 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.494895    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:35.496485 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.496460    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-5dw67" event={"ID":"43dae30a-a63d-491c-aeea-28ba9f65c85b","Type":"ContainerStarted","Data":"1220d6c5fe0f59c1ec4e1d80bce124a293de83d7a9c8b0b1b7ff59d283160c05"}
Sep 16 17:33:35.496614 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.496513    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:33:35.516305 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.516230    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq" podStartSLOduration=1.86306271 podStartE2EDuration="5.516213863s" totalImagesPullingTime="3.653151153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:33:35.5150756 +0000 UTC m=+555.507171208" watchObservedRunningTime="2026-09-16 17:33:35.516213863 +0000 UTC m=+555.508309459"
Sep 16 17:33:35.549863 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:35.549820    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-5dw67" podStartSLOduration=1.6751589569999998 podStartE2EDuration="5.549800521s" totalImagesPullingTime="3.874641564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:33:35.54945557 +0000 UTC m=+555.541551166" watchObservedRunningTime="2026-09-16 17:33:35.549800521 +0000 UTC m=+555.541896118"
Sep 16 17:33:46.505061 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:46.505033    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-2zftq"
Sep 16 17:33:56.503444 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:33:56.503410    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-5dw67"
Sep 16 17:34:19.898773 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:19.898741    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:20.005773 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:20.005743    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:20.444837 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:20.444811    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:34:20.447425 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:20.447403    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:34:52.595916 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.595878    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:52.680569 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.680536    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"]
Sep 16 17:34:52.683516 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.683498    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.686444 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.686421    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 16 17:34:52.686444 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.686429    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 16 17:34:52.686673 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.686652    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:52.687253 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.687234    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 16 17:34:52.687350 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.687317    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-82q4t\""
Sep 16 17:34:52.698311 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.698281    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"]
Sep 16 17:34:52.744265 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.744222    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.744265 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.744273    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrgw\" (UniqueName: \"kubernetes.io/projected/ed012f33-ce75-46f2-8e2c-d45ae935a42e-kube-api-access-dfrgw\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.787583 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.787550    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:52.845478 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.845447    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.845597 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.845487    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrgw\" (UniqueName: \"kubernetes.io/projected/ed012f33-ce75-46f2-8e2c-d45ae935a42e-kube-api-access-dfrgw\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.845597 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:34:52.845586    2542 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 16 17:34:52.845674 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:34:52.845656    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert podName:ed012f33-ce75-46f2-8e2c-d45ae935a42e nodeName:}" failed. No retries permitted until 2026-09-16 17:34:53.345639929 +0000 UTC m=+633.337735503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert") pod "llmisvc-controller-manager-645f9c8fd6-7qmg4" (UID: "ed012f33-ce75-46f2-8e2c-d45ae935a42e") : secret "llmisvc-webhook-server-cert" not found
Sep 16 17:34:52.854360 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.854296    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrgw\" (UniqueName: \"kubernetes.io/projected/ed012f33-ce75-46f2-8e2c-d45ae935a42e-kube-api-access-dfrgw\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:52.942099 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:52.942060    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:53.067604 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.067567    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:53.194102 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.194076    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:53.293777 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.293734    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:53.349552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.349518    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:53.351934 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.351903    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ed012f33-ce75-46f2-8e2c-d45ae935a42e-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-7qmg4\" (UID: \"ed012f33-ce75-46f2-8e2c-d45ae935a42e\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:53.355462 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.355435    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:34:53.594118 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.594036    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:53.863367 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:53.863279    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"]
Sep 16 17:34:53.867910 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:34:53.867884    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poded012f33_ce75_46f2_8e2c_d45ae935a42e.slice/crio-ae778b9030d7321a1375cf8c0583897fa9769247a19830af418bc54cade3273a WatchSource:0}: Error finding container ae778b9030d7321a1375cf8c0583897fa9769247a19830af418bc54cade3273a: Status 404 returned error can't find the container with id ae778b9030d7321a1375cf8c0583897fa9769247a19830af418bc54cade3273a
Sep 16 17:34:54.759068 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:54.759023    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4" event={"ID":"ed012f33-ce75-46f2-8e2c-d45ae935a42e","Type":"ContainerStarted","Data":"ae778b9030d7321a1375cf8c0583897fa9769247a19830af418bc54cade3273a"}
Sep 16 17:34:56.769159 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:56.769127    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4" event={"ID":"ed012f33-ce75-46f2-8e2c-d45ae935a42e","Type":"ContainerStarted","Data":"8d59307a7ccb1d11c87e64c9354e6851670909b1ec31a03d870de9cfb512bb22"}
Sep 16 17:34:56.769539 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:56.769260    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:34:56.786704 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:34:56.786650    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4" podStartSLOduration=2.074097182 podStartE2EDuration="4.786634684s" totalImagesPullingTime="2.712537502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:34:56.786474454 +0000 UTC m=+636.778570051" watchObservedRunningTime="2026-09-16 17:34:56.786634684 +0000 UTC m=+636.778730279"
Sep 16 17:35:02.517032 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:35:02.516996    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:35:02.734467 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:35:02.734435    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:35:27.774456 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:35:27.774429    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-7qmg4"
Sep 16 17:36:11.388178 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:11.388139    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:11.549131 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:11.549094    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:11.885957 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:11.885927    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:12.081670 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:12.081613    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:12.334822 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:12.334787    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:13.981278 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:13.981201    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:36:14.078763 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:36:14.078732    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:37:47.604224 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:37:47.604134    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:37:49.329201 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:37:49.329169    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:00.988016 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:00.987981    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:01.330615 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.330527    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:01.547579 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.547545    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:38:01.550669 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.550653    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.555316 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.555292    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 17:38:01.555316 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.555310    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 17:38:01.555475 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.555320    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Sep 16 17:38:01.556132 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.556116    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9bc9t\""
Sep 16 17:38:01.556238 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.556166    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 16 17:38:01.556238 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.556196    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 17:38:01.556340 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.556325    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 17:38:01.580195 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.580167    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:38:01.600207 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.600128    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:01.638276 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.638242    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.638276 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.638278    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.638481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.638356    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.638481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.638385    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.638481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.638413    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-462nq\" (UniqueName: \"kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739382    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-462nq\" (UniqueName: \"kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739605 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739433    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739605 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739464    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739605 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739526    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739605 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739554    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.739972 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.739950    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.740323 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.740304    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.740404 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.740380    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.741255 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.741230    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:01.742257 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.742236    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.752971 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.752953    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-462nq\" (UniqueName: \"kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-fbqf7\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:01.860380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:01.860291    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:02.134218 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:02.134139    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:38:02.138198 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:38:02.138164    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8c207aa_4161_47bf_b859_99af39544dee.slice/crio-89ef9750a67c790a587c30c36776661bcad47abeabea5ee8ece246f6d8ea2b84 WatchSource:0}: Error finding container 89ef9750a67c790a587c30c36776661bcad47abeabea5ee8ece246f6d8ea2b84: Status 404 returned error can't find the container with id 89ef9750a67c790a587c30c36776661bcad47abeabea5ee8ece246f6d8ea2b84
Sep 16 17:38:02.391463 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:02.391375    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerStarted","Data":"89ef9750a67c790a587c30c36776661bcad47abeabea5ee8ece246f6d8ea2b84"}
Sep 16 17:38:06.413750 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:06.413714    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerStarted","Data":"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"}
Sep 16 17:38:07.797626 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:07.797595    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:10.429215 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:10.429176    2542 generic.go:356] "Generic (PLEG): container finished" podID="b8c207aa-4161-47bf-b859-99af39544dee" containerID="66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1" exitCode=0
Sep 16 17:38:10.429639 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:10.429240    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerDied","Data":"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"}
Sep 16 17:38:19.276531 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:19.276480    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:38:23.490370 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:23.490281    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerStarted","Data":"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"}
Sep 16 17:38:25.500958 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:25.500924    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerStarted","Data":"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"}
Sep 16 17:38:25.501339 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:25.501113    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:25.501339 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:25.501161    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:25.502632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:25.502605    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:38:25.524454 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:25.524407    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podStartSLOduration=2.309420492 podStartE2EDuration="24.524390941s" totalImagesPullingTime="18.46522445s" totalInitContainerRuntime="3.749745999s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:38:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:38:25.522932516 +0000 UTC m=+845.515028111" watchObservedRunningTime="2026-09-16 17:38:25.524390941 +0000 UTC m=+845.516486537"
Sep 16 17:38:26.505517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:26.505481    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:38:31.511895 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:31.511843    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:38:31.512372 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:31.512344    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:38:41.513197 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:41.513151    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:38:51.512628 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:38:51.512590    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:39:01.512623 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:01.512584    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:39:11.512689 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:11.512651    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:39:20.466274 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:20.466196    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:39:20.467526 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:20.467505    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:39:21.513015 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:21.512975    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:39:31.512701 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:31.512662    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 16 17:39:41.392369 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.392333    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:39:41.392919 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.392573    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container" containerID="cri-o://ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182" gracePeriod=30
Sep 16 17:39:41.392919 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.392679    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kube-rbac-proxy" containerID="cri-o://b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad" gracePeriod=30
Sep 16 17:39:41.393628 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.393610    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:39:41.505939 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.505894    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.16:8643/healthz\": dial tcp 10.132.0.16:8643: connect: connection refused"
Sep 16 17:39:41.765101 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.765070    2542 generic.go:356] "Generic (PLEG): container finished" podID="b8c207aa-4161-47bf-b859-99af39544dee" containerID="b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad" exitCode=2
Sep 16 17:39:41.765275 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:41.765141    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerDied","Data":"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"}
Sep 16 17:39:46.224643 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.224618    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:39:46.300701 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.300663    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert\") pod \"b8c207aa-4161-47bf-b859-99af39544dee\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") "
Sep 16 17:39:46.300912 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.300726    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-462nq\" (UniqueName: \"kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq\") pod \"b8c207aa-4161-47bf-b859-99af39544dee\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") "
Sep 16 17:39:46.300912 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.300753    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config\") pod \"b8c207aa-4161-47bf-b859-99af39544dee\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") "
Sep 16 17:39:46.300912 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.300801    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location\") pod \"b8c207aa-4161-47bf-b859-99af39544dee\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") "
Sep 16 17:39:46.300912 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.300826    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls\") pod \"b8c207aa-4161-47bf-b859-99af39544dee\" (UID: \"b8c207aa-4161-47bf-b859-99af39544dee\") "
Sep 16 17:39:46.301181 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.301147    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b8c207aa-4161-47bf-b859-99af39544dee" (UID: "b8c207aa-4161-47bf-b859-99af39544dee"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:39:46.301288 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.301183    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b8c207aa-4161-47bf-b859-99af39544dee" (UID: "b8c207aa-4161-47bf-b859-99af39544dee"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:39:46.301288 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.301196    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b8c207aa-4161-47bf-b859-99af39544dee" (UID: "b8c207aa-4161-47bf-b859-99af39544dee"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:39:46.303013 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.302985    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq" (OuterVolumeSpecName: "kube-api-access-462nq") pod "b8c207aa-4161-47bf-b859-99af39544dee" (UID: "b8c207aa-4161-47bf-b859-99af39544dee"). InnerVolumeSpecName "kube-api-access-462nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:39:46.303096 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.303004    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b8c207aa-4161-47bf-b859-99af39544dee" (UID: "b8c207aa-4161-47bf-b859-99af39544dee"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:39:46.401762 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.401729    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b8c207aa-4161-47bf-b859-99af39544dee-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:39:46.401762 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.401762    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8c207aa-4161-47bf-b859-99af39544dee-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:39:46.401999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.401773    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:39:46.401999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.401781    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-462nq\" (UniqueName: \"kubernetes.io/projected/b8c207aa-4161-47bf-b859-99af39544dee-kube-api-access-462nq\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:39:46.401999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.401790    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b8c207aa-4161-47bf-b859-99af39544dee-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:39:46.785194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.785104    2542 generic.go:356] "Generic (PLEG): container finished" podID="b8c207aa-4161-47bf-b859-99af39544dee" containerID="ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182" exitCode=0
Sep 16 17:39:46.785194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.785168    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"
Sep 16 17:39:46.785194 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.785175    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerDied","Data":"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"}
Sep 16 17:39:46.785428 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.785235    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7" event={"ID":"b8c207aa-4161-47bf-b859-99af39544dee","Type":"ContainerDied","Data":"89ef9750a67c790a587c30c36776661bcad47abeabea5ee8ece246f6d8ea2b84"}
Sep 16 17:39:46.785428 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.785261    2542 scope.go:117] "RemoveContainer" containerID="b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"
Sep 16 17:39:46.794813 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.794787    2542 scope.go:117] "RemoveContainer" containerID="ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"
Sep 16 17:39:46.802297 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.802274    2542 scope.go:117] "RemoveContainer" containerID="66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"
Sep 16 17:39:46.809236 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.809206    2542 scope.go:117] "RemoveContainer" containerID="b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"
Sep 16 17:39:46.809565 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:39:46.809531    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad\": container with ID starting with b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad not found: ID does not exist" containerID="b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"
Sep 16 17:39:46.809663 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.809577    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad"} err="failed to get container status \"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad\": rpc error: code = NotFound desc = could not find container \"b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad\": container with ID starting with b84d54a357b0cb772d38b6b11aa2c776e48b650d1e952886ffa3460796ace9ad not found: ID does not exist"
Sep 16 17:39:46.809663 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.809601    2542 scope.go:117] "RemoveContainer" containerID="ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"
Sep 16 17:39:46.810010 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:39:46.809992    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182\": container with ID starting with ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182 not found: ID does not exist" containerID="ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"
Sep 16 17:39:46.810067 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.810015    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182"} err="failed to get container status \"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182\": rpc error: code = NotFound desc = could not find container \"ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182\": container with ID starting with ec682b082c2dff49f6af4d222ee5192af57012024308b39e76f8648afe9f3182 not found: ID does not exist"
Sep 16 17:39:46.810067 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.810036    2542 scope.go:117] "RemoveContainer" containerID="66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"
Sep 16 17:39:46.810302 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:39:46.810284    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1\": container with ID starting with 66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1 not found: ID does not exist" containerID="66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"
Sep 16 17:39:46.810350 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.810306    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1"} err="failed to get container status \"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1\": rpc error: code = NotFound desc = could not find container \"66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1\": container with ID starting with 66076f0391cf6ccd480d7e05b13ab7825de4cbfb2e83d2b8304cb7615802e5f1 not found: ID does not exist"
Sep 16 17:39:46.811134 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.811116    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:39:46.813641 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:46.813623    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-fbqf7"]
Sep 16 17:39:48.622379 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:48.622344    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8c207aa-4161-47bf-b859-99af39544dee" path="/var/lib/kubelet/pods/b8c207aa-4161-47bf-b859-99af39544dee/volumes"
Sep 16 17:39:50.909595 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:50.909535    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:51.053126 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053093    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:39:51.053330 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053319    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container"
Sep 16 17:39:51.053371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053332    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container"
Sep 16 17:39:51.053371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053344    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kube-rbac-proxy"
Sep 16 17:39:51.053371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053349    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kube-rbac-proxy"
Sep 16 17:39:51.053371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053359    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="storage-initializer"
Sep 16 17:39:51.053371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053367    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="storage-initializer"
Sep 16 17:39:51.053521 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053405    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kube-rbac-proxy"
Sep 16 17:39:51.053521 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.053413    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8c207aa-4161-47bf-b859-99af39544dee" containerName="kserve-container"
Sep 16 17:39:51.056297 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.056280    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.062411 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.062260    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\""
Sep 16 17:39:51.062729 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.062711    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9bc9t\""
Sep 16 17:39:51.062844 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.062730    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 17:39:51.062844 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.062751    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\""
Sep 16 17:39:51.062844 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.062794    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 17:39:51.063079 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.063063    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 17:39:51.063129 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.063090    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 17:39:51.074589 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.074566    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:39:51.137601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.137563    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.137601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.137603    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvclg\" (UniqueName: \"kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.137840 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.137625    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.137840 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.137703    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.137840 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.137742    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239000 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.238908    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239000 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.238973    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239000 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.238994    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239306 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.239050    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239306 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.239089    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvclg\" (UniqueName: \"kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239414 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.239351    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239650 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.239629    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.239741 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.239711    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.241504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.241483    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.248264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.248242    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvclg\" (UniqueName: \"kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg\") pod \"isvc-canary-rollback-predictor-6cb68d45bc-cdldl\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.366429 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.366388    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:51.639007 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.638833    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:39:51.641545 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:39:51.641519    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f43a0c1_7def_48fd_aed0_01ecd8982251.slice/crio-971f5cfe2159597428aa71efd5c1aa8943e852a091fdca5727a447e7e13556f7 WatchSource:0}: Error finding container 971f5cfe2159597428aa71efd5c1aa8943e852a091fdca5727a447e7e13556f7: Status 404 returned error can't find the container with id 971f5cfe2159597428aa71efd5c1aa8943e852a091fdca5727a447e7e13556f7
Sep 16 17:39:51.804364 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.804323    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerStarted","Data":"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"}
Sep 16 17:39:51.804564 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:51.804407    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerStarted","Data":"971f5cfe2159597428aa71efd5c1aa8943e852a091fdca5727a447e7e13556f7"}
Sep 16 17:39:55.820440 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:55.820405    2542 generic.go:356] "Generic (PLEG): container finished" podID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerID="b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191" exitCode=0
Sep 16 17:39:55.820927 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:55.820480    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerDied","Data":"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"}
Sep 16 17:39:55.999402 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:55.999364    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:56.124970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.124874    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:56.241121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.241089    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:56.274484 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.274451    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:39:56.277597 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.277580    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.280985 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.280964    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-7ca88-predictor-serving-cert\""
Sep 16 17:39:56.281100 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.281034    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-raw-7ca88-kube-rbac-proxy-sar-config\""
Sep 16 17:39:56.291226 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.291201    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:39:56.365659 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.365624    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:56.373291 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.373270    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.373357 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.373301    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfzx4\" (UniqueName: \"kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.373394 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.373367    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.373431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.373406    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.373463 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.373434    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.466886 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.466840    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:39:56.473765 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.473733    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.473947 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.473771    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.473947 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.473787    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.473947 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.473829    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.473947 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.473879    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfzx4\" (UniqueName: \"kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.474271 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.474252    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.474529 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.474511    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.474651 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.474632    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.476264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.476246    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.483111 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.483075    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfzx4\" (UniqueName: \"kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4\") pod \"isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.587538 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.587501    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:39:56.826206 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.826114    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerStarted","Data":"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"}
Sep 16 17:39:56.826206 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.826188    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerStarted","Data":"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"}
Sep 16 17:39:56.826711 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.826411    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:56.849736 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.849688    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podStartSLOduration=1.961030954 podStartE2EDuration="5.849673522s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.888642568s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:39:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:39:56.848018631 +0000 UTC m=+936.840114226" watchObservedRunningTime="2026-09-16 17:39:56.849673522 +0000 UTC m=+936.841769117"
Sep 16 17:39:56.859970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:56.859949    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:39:56.861981 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:39:56.861957    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47eb37b7_e114_446c_b664_d1aa00e5c778.slice/crio-931414da65fa98551e798ba029009eaa00ba52e78fb09b3fc512ac5021aecb41 WatchSource:0}: Error finding container 931414da65fa98551e798ba029009eaa00ba52e78fb09b3fc512ac5021aecb41: Status 404 returned error can't find the container with id 931414da65fa98551e798ba029009eaa00ba52e78fb09b3fc512ac5021aecb41
Sep 16 17:39:57.831026 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:57.830984    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerStarted","Data":"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"}
Sep 16 17:39:57.831451 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:57.831047    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerStarted","Data":"931414da65fa98551e798ba029009eaa00ba52e78fb09b3fc512ac5021aecb41"}
Sep 16 17:39:57.831451 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:57.831356    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:39:57.832722 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:57.832698    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:39:58.835076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:39:58.835041    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:00.842483 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:00.842448    2542 generic.go:356] "Generic (PLEG): container finished" podID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerID="e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780" exitCode=0
Sep 16 17:40:00.842906 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:00.842522    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerDied","Data":"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"}
Sep 16 17:40:00.843832 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:00.843817    2542 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 17:40:03.840293 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:03.840261    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:40:03.840816 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:03.840785    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:13.840723 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:13.840685    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:19.920818 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:19.920782    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerStarted","Data":"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"}
Sep 16 17:40:19.921264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:19.920860    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerStarted","Data":"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"}
Sep 16 17:40:19.921264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:19.921043    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:40:19.945712 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:19.945660    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podStartSLOduration=1.894930294 podStartE2EDuration="23.945646131s" totalImagesPullingTime="18.3240999s" totalInitContainerRuntime="3.726615937s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:39:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:40:19.944343295 +0000 UTC m=+959.936438889" watchObservedRunningTime="2026-09-16 17:40:19.945646131 +0000 UTC m=+959.937741727"
Sep 16 17:40:20.924793 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:20.924764    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:40:20.926204 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:20.926173    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:40:21.928173 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:21.928129    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:40:23.840749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:23.840708    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:26.933245 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:26.933206    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:40:26.933792 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:26.933764    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:40:33.841614 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:33.841572    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:36.933942 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:36.933903    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:40:37.733948 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:37.733915    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:40:43.841384 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:43.841300    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:46.934066 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:46.934024    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:40:53.841747 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:53.841708    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:40:56.934018 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:40:56.933974    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:41:03.841603 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:03.841559    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:41:06.934135 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:06.934096    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:41:07.618874 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:07.618809    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.17:8080: connect: connection refused"
Sep 16 17:41:16.938713 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:16.938622    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:41:17.619482 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:17.619446    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:41:20.997910 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:20.997877    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:41:20.998401 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:20.998134    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container" containerID="cri-o://ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4" gracePeriod=30
Sep 16 17:41:20.998401 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:20.998167    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kube-rbac-proxy" containerID="cri-o://f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123" gracePeriod=30
Sep 16 17:41:21.025356 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:21.025327    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:21.147922 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:21.147893    2542 generic.go:356] "Generic (PLEG): container finished" podID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerID="f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123" exitCode=2
Sep 16 17:41:21.148073 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:21.147959    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerDied","Data":"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"}
Sep 16 17:41:23.835452 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:23.835405    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.17:8643/healthz\": dial tcp 10.132.0.17:8643: connect: connection refused"
Sep 16 17:41:25.758440 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.758417    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:41:25.869826 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.869736    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvclg\" (UniqueName: \"kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg\") pod \"1f43a0c1-7def-48fd-aed0-01ecd8982251\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") "
Sep 16 17:41:25.869826 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.869821    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls\") pod \"1f43a0c1-7def-48fd-aed0-01ecd8982251\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") "
Sep 16 17:41:25.870046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.869936    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location\") pod \"1f43a0c1-7def-48fd-aed0-01ecd8982251\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") "
Sep 16 17:41:25.870046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.869984    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config\") pod \"1f43a0c1-7def-48fd-aed0-01ecd8982251\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") "
Sep 16 17:41:25.870046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.870034    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert\") pod \"1f43a0c1-7def-48fd-aed0-01ecd8982251\" (UID: \"1f43a0c1-7def-48fd-aed0-01ecd8982251\") "
Sep 16 17:41:25.870300 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.870267    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1f43a0c1-7def-48fd-aed0-01ecd8982251" (UID: "1f43a0c1-7def-48fd-aed0-01ecd8982251"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:41:25.870420 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.870375    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1f43a0c1-7def-48fd-aed0-01ecd8982251" (UID: "1f43a0c1-7def-48fd-aed0-01ecd8982251"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:41:25.870471 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.870424    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1f43a0c1-7def-48fd-aed0-01ecd8982251" (UID: "1f43a0c1-7def-48fd-aed0-01ecd8982251"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:41:25.872036 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.872014    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1f43a0c1-7def-48fd-aed0-01ecd8982251" (UID: "1f43a0c1-7def-48fd-aed0-01ecd8982251"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:41:25.872126 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.872106    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg" (OuterVolumeSpecName: "kube-api-access-fvclg") pod "1f43a0c1-7def-48fd-aed0-01ecd8982251" (UID: "1f43a0c1-7def-48fd-aed0-01ecd8982251"). InnerVolumeSpecName "kube-api-access-fvclg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:41:25.971473 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.971436    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:25.971473 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.971464    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvclg\" (UniqueName: \"kubernetes.io/projected/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-api-access-fvclg\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:25.971473 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.971474    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1f43a0c1-7def-48fd-aed0-01ecd8982251-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:25.971473 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.971483    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1f43a0c1-7def-48fd-aed0-01ecd8982251-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:25.971734 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:25.971493    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1f43a0c1-7def-48fd-aed0-01ecd8982251-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:26.163331 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.163285    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:26.167600 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.167574    2542 generic.go:356] "Generic (PLEG): container finished" podID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerID="ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4" exitCode=0
Sep 16 17:41:26.167733 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.167617    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerDied","Data":"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"}
Sep 16 17:41:26.167733 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.167625    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"
Sep 16 17:41:26.167733 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.167666    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl" event={"ID":"1f43a0c1-7def-48fd-aed0-01ecd8982251","Type":"ContainerDied","Data":"971f5cfe2159597428aa71efd5c1aa8943e852a091fdca5727a447e7e13556f7"}
Sep 16 17:41:26.167733 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.167694    2542 scope.go:117] "RemoveContainer" containerID="f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"
Sep 16 17:41:26.176157 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.176138    2542 scope.go:117] "RemoveContainer" containerID="ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"
Sep 16 17:41:26.183435 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.183420    2542 scope.go:117] "RemoveContainer" containerID="b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"
Sep 16 17:41:26.190645 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.190629    2542 scope.go:117] "RemoveContainer" containerID="f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"
Sep 16 17:41:26.190936 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:26.190913    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123\": container with ID starting with f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123 not found: ID does not exist" containerID="f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"
Sep 16 17:41:26.190991 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.190948    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123"} err="failed to get container status \"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123\": rpc error: code = NotFound desc = could not find container \"f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123\": container with ID starting with f5259503ce50648dc5c9028967436ffe46b805b3943110358aaf1047635c9123 not found: ID does not exist"
Sep 16 17:41:26.190991 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.190965    2542 scope.go:117] "RemoveContainer" containerID="ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"
Sep 16 17:41:26.191239 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:26.191220    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4\": container with ID starting with ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4 not found: ID does not exist" containerID="ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"
Sep 16 17:41:26.191294 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.191247    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4"} err="failed to get container status \"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4\": rpc error: code = NotFound desc = could not find container \"ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4\": container with ID starting with ea463589dba9f598c7cc5d15e1994dc6ab7bf12af00bdbc13211d6c0cd2d00d4 not found: ID does not exist"
Sep 16 17:41:26.191294 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.191262    2542 scope.go:117] "RemoveContainer" containerID="b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"
Sep 16 17:41:26.191531 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:26.191513    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191\": container with ID starting with b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191 not found: ID does not exist" containerID="b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"
Sep 16 17:41:26.191583 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.191538    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191"} err="failed to get container status \"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191\": rpc error: code = NotFound desc = could not find container \"b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191\": container with ID starting with b27b381874d6375c8153c6df28f92ed7391eb080c46545e67f9ea8bd39075191 not found: ID does not exist"
Sep 16 17:41:26.196630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.196610    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:41:26.201935 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.201913    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-6cb68d45bc-cdldl"]
Sep 16 17:41:26.259653 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.259613    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:26.487708 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.487621    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:26.622653 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.622620    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" path="/var/lib/kubelet/pods/1f43a0c1-7def-48fd-aed0-01ecd8982251/volumes"
Sep 16 17:41:26.934041 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:26.934008    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:41:36.115811 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.115779    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116138    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="storage-initializer"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116151    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="storage-initializer"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116161    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116167    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116181    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kube-rbac-proxy"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116186    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kube-rbac-proxy"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116226    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kube-rbac-proxy"
Sep 16 17:41:36.116292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.116235    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f43a0c1-7def-48fd-aed0-01ecd8982251" containerName="kserve-container"
Sep 16 17:41:36.120316 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.120298    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.122504 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.122482    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-7ca88-kube-rbac-proxy-sar-config\""
Sep 16 17:41:36.122628 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.122486    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-7ca88-serving-cert\""
Sep 16 17:41:36.128082 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.128059    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:41:36.128266 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.128242    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:36.244172 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.244133    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.244172 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.244174    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.315564 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.315521    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:36.344681 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.344645    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.344681 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.344681    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.345435 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.345405    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.347232 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.347210    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls\") pod \"model-chainer-raw-7ca88-568dd86d44-f8wrn\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") " pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.431337 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.431306    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:36.701192 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:36.701007    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:41:36.703979 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:41:36.703950    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fab8029_1c56_4cca_8924_71aaa7d49772.slice/crio-e9f63e54785056a31477bd1b34d1f8346bb505766f9f7185d00be4aed01fa59f WatchSource:0}: Error finding container e9f63e54785056a31477bd1b34d1f8346bb505766f9f7185d00be4aed01fa59f: Status 404 returned error can't find the container with id e9f63e54785056a31477bd1b34d1f8346bb505766f9f7185d00be4aed01fa59f
Sep 16 17:41:37.209166 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:37.209122    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" event={"ID":"7fab8029-1c56-4cca-8924-71aaa7d49772","Type":"ContainerStarted","Data":"e9f63e54785056a31477bd1b34d1f8346bb505766f9f7185d00be4aed01fa59f"}
Sep 16 17:41:39.217485 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:39.217391    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" event={"ID":"7fab8029-1c56-4cca-8924-71aaa7d49772","Type":"ContainerStarted","Data":"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"}
Sep 16 17:41:39.217906 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:39.217504    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:39.237923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:39.237880    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podStartSLOduration=1.043737506 podStartE2EDuration="3.237866229s" totalImagesPullingTime="2.194128723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:41:39.235445904 +0000 UTC m=+1039.227541511" watchObservedRunningTime="2026-09-16 17:41:39.237866229 +0000 UTC m=+1039.229961816"
Sep 16 17:41:45.226530 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:45.226492    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:41:46.179206 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.179173    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:41:46.179410 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.179355    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" containerID="cri-o://466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23" gracePeriod=30
Sep 16 17:41:46.210008 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.209959    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:46.317543 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.317506    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:46.331007 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.330977    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:41:46.331221 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.331200    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" containerID="cri-o://85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043" gracePeriod=30
Sep 16 17:41:46.331306 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.331255    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kube-rbac-proxy" containerID="cri-o://12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f" gracePeriod=30
Sep 16 17:41:46.410494 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.410458    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:46.502699 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.502660    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:46.639808 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.639775    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:46.929149 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.929115    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.18:8643/healthz\": dial tcp 10.132.0.18:8643: connect: connection refused"
Sep 16 17:41:46.934532 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.934505    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.18:8080: connect: connection refused"
Sep 16 17:41:46.980072 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:46.980032    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:47.248264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:47.248174    2542 generic.go:356] "Generic (PLEG): container finished" podID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerID="12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f" exitCode=2
Sep 16 17:41:47.248264 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:47.248252    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerDied","Data":"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"}
Sep 16 17:41:47.341645 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:47.341606    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:41:49.978614 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:49.978589    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:41:50.152627 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.152588    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config\") pod \"47eb37b7-e114-446c-b664-d1aa00e5c778\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") "
Sep 16 17:41:50.152801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.152651    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert\") pod \"47eb37b7-e114-446c-b664-d1aa00e5c778\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") "
Sep 16 17:41:50.152801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.152681    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jfzx4\" (UniqueName: \"kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4\") pod \"47eb37b7-e114-446c-b664-d1aa00e5c778\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") "
Sep 16 17:41:50.152801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.152700    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls\") pod \"47eb37b7-e114-446c-b664-d1aa00e5c778\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") "
Sep 16 17:41:50.152801 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.152718    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location\") pod \"47eb37b7-e114-446c-b664-d1aa00e5c778\" (UID: \"47eb37b7-e114-446c-b664-d1aa00e5c778\") "
Sep 16 17:41:50.153150 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.153022    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "47eb37b7-e114-446c-b664-d1aa00e5c778" (UID: "47eb37b7-e114-446c-b664-d1aa00e5c778"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:41:50.153230 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.153124    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "47eb37b7-e114-446c-b664-d1aa00e5c778" (UID: "47eb37b7-e114-446c-b664-d1aa00e5c778"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:41:50.153230 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.153173    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "47eb37b7-e114-446c-b664-d1aa00e5c778" (UID: "47eb37b7-e114-446c-b664-d1aa00e5c778"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:41:50.154743 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.154719    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "47eb37b7-e114-446c-b664-d1aa00e5c778" (UID: "47eb37b7-e114-446c-b664-d1aa00e5c778"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:41:50.154834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.154818    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4" (OuterVolumeSpecName: "kube-api-access-jfzx4") pod "47eb37b7-e114-446c-b664-d1aa00e5c778" (UID: "47eb37b7-e114-446c-b664-d1aa00e5c778"). InnerVolumeSpecName "kube-api-access-jfzx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:41:50.225673 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.225634    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:41:50.254214 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.254183    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:50.254214 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.254212    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/47eb37b7-e114-446c-b664-d1aa00e5c778-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:50.254378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.254222    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jfzx4\" (UniqueName: \"kubernetes.io/projected/47eb37b7-e114-446c-b664-d1aa00e5c778-kube-api-access-jfzx4\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:50.254378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.254233    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/47eb37b7-e114-446c-b664-d1aa00e5c778-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:50.254378 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.254243    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/47eb37b7-e114-446c-b664-d1aa00e5c778-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:41:50.261328 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.261296    2542 generic.go:356] "Generic (PLEG): container finished" podID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerID="85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043" exitCode=0
Sep 16 17:41:50.261439 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.261358    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"
Sep 16 17:41:50.261439 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.261382    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerDied","Data":"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"}
Sep 16 17:41:50.261509 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.261445    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8" event={"ID":"47eb37b7-e114-446c-b664-d1aa00e5c778","Type":"ContainerDied","Data":"931414da65fa98551e798ba029009eaa00ba52e78fb09b3fc512ac5021aecb41"}
Sep 16 17:41:50.261509 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.261475    2542 scope.go:117] "RemoveContainer" containerID="12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"
Sep 16 17:41:50.270021 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.270004    2542 scope.go:117] "RemoveContainer" containerID="85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"
Sep 16 17:41:50.276833 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.276817    2542 scope.go:117] "RemoveContainer" containerID="e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"
Sep 16 17:41:50.282742 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.282718    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:41:50.283818 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.283804    2542 scope.go:117] "RemoveContainer" containerID="12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"
Sep 16 17:41:50.284144 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:50.284124    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f\": container with ID starting with 12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f not found: ID does not exist" containerID="12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"
Sep 16 17:41:50.284255 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.284152    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f"} err="failed to get container status \"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f\": rpc error: code = NotFound desc = could not find container \"12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f\": container with ID starting with 12daf6a8b4062ca9d460fbf93c92c02d1ff42ce47c3939abc79ab6996b62720f not found: ID does not exist"
Sep 16 17:41:50.284255 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.284169    2542 scope.go:117] "RemoveContainer" containerID="85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"
Sep 16 17:41:50.284418 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:50.284401    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043\": container with ID starting with 85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043 not found: ID does not exist" containerID="85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"
Sep 16 17:41:50.284463 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.284426    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043"} err="failed to get container status \"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043\": rpc error: code = NotFound desc = could not find container \"85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043\": container with ID starting with 85ed7bf93444ec8e53c1035cd93d50a3c9b90353b473ec56bb5ff6e39220f043 not found: ID does not exist"
Sep 16 17:41:50.284463 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.284441    2542 scope.go:117] "RemoveContainer" containerID="e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"
Sep 16 17:41:50.284689 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:41:50.284672    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780\": container with ID starting with e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780 not found: ID does not exist" containerID="e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"
Sep 16 17:41:50.284730 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.284693    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780"} err="failed to get container status \"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780\": rpc error: code = NotFound desc = could not find container \"e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780\": container with ID starting with e29448fe495f86ae08bb659d314fc7641205103eb5b120bd31c1ea5aea834780 not found: ID does not exist"
Sep 16 17:41:50.286691 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.286669    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-raw-7ca88-predictor-68cc87ffd9-n92q8"]
Sep 16 17:41:50.622601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:50.622523    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" path="/var/lib/kubelet/pods/47eb37b7-e114-446c-b664-d1aa00e5c778/volumes"
Sep 16 17:41:55.226429 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:41:55.226385    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:42:00.225577 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:00.225539    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:42:00.226050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:00.225640    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:42:05.225211 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:05.225170    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:42:10.225634 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:10.225595    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:42:15.225452 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:15.225365    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:42:16.315586 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.315559    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:42:16.339039 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.339009    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle\") pod \"7fab8029-1c56-4cca-8924-71aaa7d49772\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") "
Sep 16 17:42:16.339220 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.339107    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls\") pod \"7fab8029-1c56-4cca-8924-71aaa7d49772\" (UID: \"7fab8029-1c56-4cca-8924-71aaa7d49772\") "
Sep 16 17:42:16.339438 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.339409    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "7fab8029-1c56-4cca-8924-71aaa7d49772" (UID: "7fab8029-1c56-4cca-8924-71aaa7d49772"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:42:16.341102 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.341079    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7fab8029-1c56-4cca-8924-71aaa7d49772" (UID: "7fab8029-1c56-4cca-8924-71aaa7d49772"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:42:16.351123 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.351088    2542 generic.go:356] "Generic (PLEG): container finished" podID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerID="466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23" exitCode=0
Sep 16 17:42:16.351219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.351124    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" event={"ID":"7fab8029-1c56-4cca-8924-71aaa7d49772","Type":"ContainerDied","Data":"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"}
Sep 16 17:42:16.351219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.351132    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"
Sep 16 17:42:16.351219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.351163    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn" event={"ID":"7fab8029-1c56-4cca-8924-71aaa7d49772","Type":"ContainerDied","Data":"e9f63e54785056a31477bd1b34d1f8346bb505766f9f7185d00be4aed01fa59f"}
Sep 16 17:42:16.351219 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.351185    2542 scope.go:117] "RemoveContainer" containerID="466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"
Sep 16 17:42:16.359003 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.358985    2542 scope.go:117] "RemoveContainer" containerID="466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"
Sep 16 17:42:16.359312 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:42:16.359291    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23\": container with ID starting with 466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23 not found: ID does not exist" containerID="466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"
Sep 16 17:42:16.359381 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.359325    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23"} err="failed to get container status \"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23\": rpc error: code = NotFound desc = could not find container \"466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23\": container with ID starting with 466b2563088dc09e8585c52a7692f85d0e210af5cb813abc0aed681accddaf23 not found: ID does not exist"
Sep 16 17:42:16.377132 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.377105    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:42:16.384584 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.384563    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-7ca88-568dd86d44-f8wrn"]
Sep 16 17:42:16.439882 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.439787    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7fab8029-1c56-4cca-8924-71aaa7d49772-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:42:16.439882 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.439815    2542 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7fab8029-1c56-4cca-8924-71aaa7d49772-openshift-service-ca-bundle\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:42:16.623250 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:16.623218    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" path="/var/lib/kubelet/pods/7fab8029-1c56-4cca-8924-71aaa7d49772/volumes"
Sep 16 17:42:22.836627 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:22.836582    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:42:56.207921 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:56.207877    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:42:57.532922 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:42:57.532882    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:06.414417 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.414378    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:06.445985 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.445947    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:43:06.446239 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446226    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="storage-initializer"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446243    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="storage-initializer"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446255    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kube-rbac-proxy"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446260    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kube-rbac-proxy"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446267    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446273    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446281    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88"
Sep 16 17:43:06.446296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446286    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88"
Sep 16 17:43:06.446535 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446334    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fab8029-1c56-4cca-8924-71aaa7d49772" containerName="model-chainer-raw-7ca88"
Sep 16 17:43:06.446535 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446341    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kserve-container"
Sep 16 17:43:06.446535 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.446347    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="47eb37b7-e114-446c-b664-d1aa00e5c778" containerName="kube-rbac-proxy"
Sep 16 17:43:06.449354 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.449338    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.451904 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.451878    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-9501e1-predictor-serving-cert\""
Sep 16 17:43:06.452027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.451954    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 17:43:06.452093 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.452023    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9bc9t\""
Sep 16 17:43:06.452093 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.452046    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 17:43:06.452912 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.452897    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 17:43:06.453002 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.452925    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 17:43:06.453002 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.452902    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-primary-9501e1-kube-rbac-proxy-sar-config\""
Sep 16 17:43:06.460420 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.460396    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:43:06.589558 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.589521    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.589755 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.589572    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.589755 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.589593    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.589755 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.589639    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j94bm\" (UniqueName: \"kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.589755 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.589714    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691042 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.690949    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691042 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691013    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691291 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691042    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691291 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691068    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j94bm\" (UniqueName: \"kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691291 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691094    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691605 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691583    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691783 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691762    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.691824 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.691762    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.693627 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.693601    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.703410 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.703382    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j94bm\" (UniqueName: \"kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm\") pod \"isvc-primary-9501e1-predictor-844c64b999-ndrcv\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") " pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:06.731507 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.731477    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:06.759546 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:06.759520    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:07.033681 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:07.033618    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:43:07.036212 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:43:07.036177    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6cc3f8_0de3_4cd4_9e83_d453f4ec8ae7.slice/crio-e637332ba46bd2cc0d4904e2aa9248a914837b325b16539aa8c02798951d3b4d WatchSource:0}: Error finding container e637332ba46bd2cc0d4904e2aa9248a914837b325b16539aa8c02798951d3b4d: Status 404 returned error can't find the container with id e637332ba46bd2cc0d4904e2aa9248a914837b325b16539aa8c02798951d3b4d
Sep 16 17:43:07.522015 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:07.521980    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerStarted","Data":"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"}
Sep 16 17:43:07.522502 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:07.522040    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerStarted","Data":"e637332ba46bd2cc0d4904e2aa9248a914837b325b16539aa8c02798951d3b4d"}
Sep 16 17:43:11.536982 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:11.536948    2542 generic.go:356] "Generic (PLEG): container finished" podID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerID="c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff" exitCode=0
Sep 16 17:43:11.537460 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:11.537024    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerDied","Data":"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"}
Sep 16 17:43:12.543642 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:12.543604    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerStarted","Data":"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"}
Sep 16 17:43:12.544033 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:12.543669    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerStarted","Data":"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"}
Sep 16 17:43:12.544033 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:12.543883    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:12.573177 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:12.573124    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podStartSLOduration=2.647230539 podStartE2EDuration="6.573107942s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.925877403s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:43:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:43:12.572472276 +0000 UTC m=+1132.564567885" watchObservedRunningTime="2026-09-16 17:43:12.573107942 +0000 UTC m=+1132.565203609"
Sep 16 17:43:13.547465 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:13.547433    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:13.548911 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:13.548882    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:14.550296 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:14.550259    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:15.976730 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:15.976695    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:19.555243 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:19.555214    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:43:19.555821 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:19.555795    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:29.555749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:29.555710    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:36.474713 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.474674    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:43:36.477749 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.477732    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.480613 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.480595    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-c0e2d-kube-rbac-proxy-sar-config\""
Sep 16 17:43:36.480672 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.480618    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-c0e2d-serving-cert\""
Sep 16 17:43:36.494190 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.494167    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:43:36.505982 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.505956    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:36.613965 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.613928    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.614141 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.613976    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.714464 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.714426    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.714642 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.714477    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.714642 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:43:36.714571    2542 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-serving-cert: secret "model-chainer-raw-hpa-c0e2d-serving-cert" not found
Sep 16 17:43:36.714642 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:43:36.714635    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls podName:ed51d252-e061-4a67-866f-de359f655c5a nodeName:}" failed. No retries permitted until 2026-09-16 17:43:37.214617669 +0000 UTC m=+1157.206713243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls") pod "model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" (UID: "ed51d252-e061-4a67-866f-de359f655c5a") : secret "model-chainer-raw-hpa-c0e2d-serving-cert" not found
Sep 16 17:43:36.715072 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.715052    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:36.860032 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:36.859942    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:37.218159 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:37.218118    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:37.220635 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:37.220612    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") pod \"model-chainer-raw-hpa-c0e2d-7884755577-qjpc4\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:37.387613 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:37.387574    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:37.655785 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:37.655588    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:43:37.658543 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:43:37.658518    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded51d252_e061_4a67_866f_de359f655c5a.slice/crio-e6782c73bba43e7091ba9de522bee4e2e80a1fc6de70c3cdebfa1162336cbfbc WatchSource:0}: Error finding container e6782c73bba43e7091ba9de522bee4e2e80a1fc6de70c3cdebfa1162336cbfbc: Status 404 returned error can't find the container with id e6782c73bba43e7091ba9de522bee4e2e80a1fc6de70c3cdebfa1162336cbfbc
Sep 16 17:43:38.634518 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:38.634484    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" event={"ID":"ed51d252-e061-4a67-866f-de359f655c5a","Type":"ContainerStarted","Data":"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"}
Sep 16 17:43:38.634724 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:38.634534    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" event={"ID":"ed51d252-e061-4a67-866f-de359f655c5a","Type":"ContainerStarted","Data":"e6782c73bba43e7091ba9de522bee4e2e80a1fc6de70c3cdebfa1162336cbfbc"}
Sep 16 17:43:38.634724 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:38.634615    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:38.651466 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:38.651417    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podStartSLOduration=2.651402377 podStartE2EDuration="2.651402377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:43:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:43:38.650575597 +0000 UTC m=+1158.642671217" watchObservedRunningTime="2026-09-16 17:43:38.651402377 +0000 UTC m=+1158.643497972"
Sep 16 17:43:39.556088 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:39.556050    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:44.644384 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:44.644306    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:43:46.526948 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.526917    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:43:46.527397 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.527095    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" containerID="cri-o://dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d" gracePeriod=30
Sep 16 17:43:46.551523 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.551491    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:46.681637 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.681594    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:46.791015 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.790918    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:46.965240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:46.965207    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:43:49.555827 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:49.555783    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:49.642500 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:49.642466    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:43:54.643494 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:54.643452    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:43:59.556313 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:59.556274    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:43:59.642305 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:59.642271    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:43:59.642473 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:43:59.642364    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:44:04.642552 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:04.642513    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:44:06.783337 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:06.783288    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:44:07.021595 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:07.021558    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:44:09.556721 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:09.556679    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:44:09.643369 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:09.643331    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:44:14.643051 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:14.643013    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 17:44:16.666515 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.666487    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:44:16.690455 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.690431    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle\") pod \"ed51d252-e061-4a67-866f-de359f655c5a\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") "
Sep 16 17:44:16.690575 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.690472    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") pod \"ed51d252-e061-4a67-866f-de359f655c5a\" (UID: \"ed51d252-e061-4a67-866f-de359f655c5a\") "
Sep 16 17:44:16.690896 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.690841    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ed51d252-e061-4a67-866f-de359f655c5a" (UID: "ed51d252-e061-4a67-866f-de359f655c5a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:44:16.693191 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.693154    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ed51d252-e061-4a67-866f-de359f655c5a" (UID: "ed51d252-e061-4a67-866f-de359f655c5a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:44:16.769222 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.769186    2542 generic.go:356] "Generic (PLEG): container finished" podID="ed51d252-e061-4a67-866f-de359f655c5a" containerID="dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d" exitCode=0
Sep 16 17:44:16.769222 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.769225    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" event={"ID":"ed51d252-e061-4a67-866f-de359f655c5a","Type":"ContainerDied","Data":"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"}
Sep 16 17:44:16.769424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.769241    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"
Sep 16 17:44:16.769424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.769267    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4" event={"ID":"ed51d252-e061-4a67-866f-de359f655c5a","Type":"ContainerDied","Data":"e6782c73bba43e7091ba9de522bee4e2e80a1fc6de70c3cdebfa1162336cbfbc"}
Sep 16 17:44:16.769424 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.769289    2542 scope.go:117] "RemoveContainer" containerID="dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"
Sep 16 17:44:16.777395 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.777370    2542 scope.go:117] "RemoveContainer" containerID="dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"
Sep 16 17:44:16.777673 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:44:16.777655    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d\": container with ID starting with dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d not found: ID does not exist" containerID="dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"
Sep 16 17:44:16.777721 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.777681    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d"} err="failed to get container status \"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d\": rpc error: code = NotFound desc = could not find container \"dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d\": container with ID starting with dd16c91729f47c5af70c53a24639ef3fcacd30b96edac5df80a71e89b3d8cc5d not found: ID does not exist"
Sep 16 17:44:16.791769 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.791716    2542 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed51d252-e061-4a67-866f-de359f655c5a-openshift-service-ca-bundle\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:44:16.791769 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.791746    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ed51d252-e061-4a67-866f-de359f655c5a-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:44:16.798023 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.798002    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:44:16.801778 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:16.801756    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-c0e2d-7884755577-qjpc4"]
Sep 16 17:44:18.622276 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:18.622243    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed51d252-e061-4a67-866f-de359f655c5a" path="/var/lib/kubelet/pods/ed51d252-e061-4a67-866f-de359f655c5a/volumes"
Sep 16 17:44:19.556481 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:19.556440    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.20:8080: connect: connection refused"
Sep 16 17:44:20.488731 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:20.488704    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:44:20.489174 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:20.489093    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:44:29.557031 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:29.557001    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:44:51.609214 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:51.609178    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:44:52.593731 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:44:52.593701    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:11.716826 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.716772    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:11.728377 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.728343    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:45:11.728666 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.728635    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container" containerID="cri-o://050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e" gracePeriod=30
Sep 16 17:45:11.728810 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.728669    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kube-rbac-proxy" containerID="cri-o://9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42" gracePeriod=30
Sep 16 17:45:11.784127 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.784088    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:11.971263 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.971167    2542 generic.go:356] "Generic (PLEG): container finished" podID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerID="9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42" exitCode=2
Sep 16 17:45:11.971263 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:11.971232    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerDied","Data":"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"}
Sep 16 17:45:14.551056 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:14.550960    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.20:8643/healthz\": dial tcp 10.132.0.20:8643: connect: connection refused"
Sep 16 17:45:16.666456 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.666430    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:45:16.728459 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728425    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config\") pod \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") "
Sep 16 17:45:16.728639 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728473    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls\") pod \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") "
Sep 16 17:45:16.728639 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728507    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location\") pod \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") "
Sep 16 17:45:16.728741 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728681    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j94bm\" (UniqueName: \"kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm\") pod \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") "
Sep 16 17:45:16.728741 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728723    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert\") pod \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\" (UID: \"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7\") "
Sep 16 17:45:16.728815 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728784    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" (UID: "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:45:16.728815 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.728787    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" (UID: "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:45:16.729046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.729024    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:45:16.729046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.729046    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:45:16.729240 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.729079    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" (UID: "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:45:16.731188 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.731169    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" (UID: "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:45:16.731251 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.731190    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm" (OuterVolumeSpecName: "kube-api-access-j94bm") pod "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" (UID: "4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7"). InnerVolumeSpecName "kube-api-access-j94bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:45:16.829834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.829747    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:45:16.829834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.829777    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j94bm\" (UniqueName: \"kubernetes.io/projected/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-kube-api-access-j94bm\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:45:16.829834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.829787    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:45:16.981558 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.981525    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:16.990865 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.990814    2542 generic.go:356] "Generic (PLEG): container finished" podID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerID="050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e" exitCode=0
Sep 16 17:45:16.990999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.990891    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"
Sep 16 17:45:16.990999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.990889    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerDied","Data":"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"}
Sep 16 17:45:16.990999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.990951    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv" event={"ID":"4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7","Type":"ContainerDied","Data":"e637332ba46bd2cc0d4904e2aa9248a914837b325b16539aa8c02798951d3b4d"}
Sep 16 17:45:16.990999 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.990979    2542 scope.go:117] "RemoveContainer" containerID="9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"
Sep 16 17:45:16.999003 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:16.998984    2542 scope.go:117] "RemoveContainer" containerID="050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"
Sep 16 17:45:17.005886 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.005862    2542 scope.go:117] "RemoveContainer" containerID="c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"
Sep 16 17:45:17.013054 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013035    2542 scope.go:117] "RemoveContainer" containerID="9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"
Sep 16 17:45:17.013295 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:45:17.013279    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42\": container with ID starting with 9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42 not found: ID does not exist" containerID="9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"
Sep 16 17:45:17.013345 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013303    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42"} err="failed to get container status \"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42\": rpc error: code = NotFound desc = could not find container \"9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42\": container with ID starting with 9b67a2ff126ce5a6cce2b09cabff12234048d7d1acf7fa85be1092c3eb79ed42 not found: ID does not exist"
Sep 16 17:45:17.013345 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013324    2542 scope.go:117] "RemoveContainer" containerID="050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"
Sep 16 17:45:17.013575 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:45:17.013561    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e\": container with ID starting with 050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e not found: ID does not exist" containerID="050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"
Sep 16 17:45:17.013622 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013578    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e"} err="failed to get container status \"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e\": rpc error: code = NotFound desc = could not find container \"050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e\": container with ID starting with 050a7e94fa113f7e9b87e203e83052b00e51f2c2e1783009b82d058a76197e1e not found: ID does not exist"
Sep 16 17:45:17.013622 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013589    2542 scope.go:117] "RemoveContainer" containerID="c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"
Sep 16 17:45:17.013798 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:45:17.013782    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff\": container with ID starting with c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff not found: ID does not exist" containerID="c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"
Sep 16 17:45:17.013834 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.013802    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff"} err="failed to get container status \"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff\": rpc error: code = NotFound desc = could not find container \"c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff\": container with ID starting with c32e5c0218e7137e9fcc0a2962471a85e858db6ac0eb6709c5ec1672495ae2ff not found: ID does not exist"
Sep 16 17:45:17.036921 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.036893    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:45:17.047155 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.047132    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-primary-9501e1-predictor-844c64b999-ndrcv"]
Sep 16 17:45:17.876982 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:17.876945    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:18.622134 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:18.622099    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" path="/var/lib/kubelet/pods/4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7/volumes"
Sep 16 17:45:27.027076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:27.027034    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:27.884205 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:27.884161    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:27.965170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:27.965073    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:32.226722 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.226670    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:32.264338 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264303    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:45:32.264606 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264590    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264610    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264630    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="storage-initializer"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264639    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="storage-initializer"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264652    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264663    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264678    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kube-rbac-proxy"
Sep 16 17:45:32.264719 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264685    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kube-rbac-proxy"
Sep 16 17:45:32.265076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264769    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kserve-container"
Sep 16 17:45:32.265076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264782    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d6cc3f8-0de3-4cd4-9e83-d453f4ec8ae7" containerName="kube-rbac-proxy"
Sep 16 17:45:32.265076 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.264790    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed51d252-e061-4a67-866f-de359f655c5a" containerName="model-chainer-raw-hpa-c0e2d"
Sep 16 17:45:32.269138 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.269116    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.274245 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.274218    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 17:45:32.274380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.274244    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 17:45:32.274380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.274252    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 17:45:32.275117 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.275100    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 17:45:32.275262 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.275247    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-7c42d-kube-rbac-proxy-sar-config\""
Sep 16 17:45:32.275324 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.275313    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"raw-sklearn-7c42d-predictor-serving-cert\""
Sep 16 17:45:32.275387 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.275373    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9bc9t\""
Sep 16 17:45:32.278648 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.278624    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:45:32.326677 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.326633    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:32.344760 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.344729    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.344963 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.344769    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2g6z\" (UniqueName: \"kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.344963 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.344807    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.344963 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.344869    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.344963 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.344900    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.445873 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.445822    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446098 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.445890    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2g6z\" (UniqueName: \"kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446098 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.445932    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446098 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.445960    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446098 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.445990    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446449 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.446426    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446596 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.446576    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.446677 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.446661    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.448396 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.448371    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.455477 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.455451    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2g6z\" (UniqueName: \"kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z\") pod \"raw-sklearn-7c42d-predictor-657dfb5546-2pxvn\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") " pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.584430 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.584335    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:32.859644 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:32.859620    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:45:32.861863 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:45:32.861825    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde03e6cd_f7f2_4ee6_9f1d_95a188eac476.slice/crio-979a51c23f4a254b461a829e3850e02c31d8cdcf226b73d7d7c934cc0c277b15 WatchSource:0}: Error finding container 979a51c23f4a254b461a829e3850e02c31d8cdcf226b73d7d7c934cc0c277b15: Status 404 returned error can't find the container with id 979a51c23f4a254b461a829e3850e02c31d8cdcf226b73d7d7c934cc0c277b15
Sep 16 17:45:33.049939 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:33.049903    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerStarted","Data":"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"}
Sep 16 17:45:33.050121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:33.049968    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerStarted","Data":"979a51c23f4a254b461a829e3850e02c31d8cdcf226b73d7d7c934cc0c277b15"}
Sep 16 17:45:37.067249 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:37.067215    2542 generic.go:356] "Generic (PLEG): container finished" podID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerID="dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5" exitCode=0
Sep 16 17:45:37.067630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:37.067287    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerDied","Data":"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"}
Sep 16 17:45:38.072883 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:38.072829    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerStarted","Data":"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"}
Sep 16 17:45:38.073256 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:38.072939    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerStarted","Data":"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"}
Sep 16 17:45:38.073256 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:38.073152    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:38.110507 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:38.110450    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podStartSLOduration=2.52281953 podStartE2EDuration="6.110436255s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.587616725s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:45:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:45:38.109901169 +0000 UTC m=+1278.101996761" watchObservedRunningTime="2026-09-16 17:45:38.110436255 +0000 UTC m=+1278.102531850"
Sep 16 17:45:39.076455 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:39.076422    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:39.077777 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:39.077741    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:45:40.080354 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:40.080318    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:45:42.044313 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.044279    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:42.154615 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.154577    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:42.194914 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.194872    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:45:42.198879 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.198842    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.202385 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.202357    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-be77d-predictor-serving-cert\""
Sep 16 17:45:42.202527 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.202503    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-scale-raw-be77d-kube-rbac-proxy-sar-config\""
Sep 16 17:45:42.216172 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.216145    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:45:42.283547 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.283514    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:42.319357 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.319256    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.319357 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.319292    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.319357 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.319309    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.319604 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.319438    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzjgf\" (UniqueName: \"kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.319604 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.319486    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.420426 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420322    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.420426 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420413    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzjgf\" (UniqueName: \"kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.420646 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420456    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.420646 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:45:42.420568    2542 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-serving-cert: secret "isvc-sklearn-scale-raw-be77d-predictor-serving-cert" not found
Sep 16 17:45:42.420646 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420589    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.420646 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:45:42.420621    2542 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls podName:96331fdb-8031-4d57-b1f6-8bde90653823 nodeName:}" failed. No retries permitted until 2026-09-16 17:45:42.920606086 +0000 UTC m=+1282.912701660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls") pod "isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823") : secret "isvc-sklearn-scale-raw-be77d-predictor-serving-cert" not found
Sep 16 17:45:42.420646 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420639    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.421007 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.420992    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.421235 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.421214    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.421272 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.421214    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.432987 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.432963    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzjgf\" (UniqueName: \"kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.924139 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.924106    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.926533 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.926508    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") pod \"isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") " pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:42.941515 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:42.941489    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:45:43.108955 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:43.108919    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:43.379903 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:43.379809    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:45:43.383147 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:45:43.383115    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96331fdb_8031_4d57_b1f6_8bde90653823.slice/crio-c5d0418f5c0cca54a9287550e3102d1ef23c1d3651076e1c0e87286b7178e693 WatchSource:0}: Error finding container c5d0418f5c0cca54a9287550e3102d1ef23c1d3651076e1c0e87286b7178e693: Status 404 returned error can't find the container with id c5d0418f5c0cca54a9287550e3102d1ef23c1d3651076e1c0e87286b7178e693
Sep 16 17:45:44.098289 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:44.098254    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerStarted","Data":"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"}
Sep 16 17:45:44.098462 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:44.098312    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerStarted","Data":"c5d0418f5c0cca54a9287550e3102d1ef23c1d3651076e1c0e87286b7178e693"}
Sep 16 17:45:45.085161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:45.085132    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:45:45.085784 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:45.085757    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:45:48.115629 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:48.115597    2542 generic.go:356] "Generic (PLEG): container finished" podID="96331fdb-8031-4d57-b1f6-8bde90653823" containerID="81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf" exitCode=0
Sep 16 17:45:48.116061 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:48.115680    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerDied","Data":"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"}
Sep 16 17:45:49.121103 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.121070    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerStarted","Data":"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"}
Sep 16 17:45:49.121522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.121130    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerStarted","Data":"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"}
Sep 16 17:45:49.121522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.121412    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:49.121522 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.121473    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:49.122817 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.122789    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:45:49.144610 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:49.144568    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podStartSLOduration=3.296913435 podStartE2EDuration="7.144553166s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.847639731s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:45:49.14334349 +0000 UTC m=+1289.135439110" watchObservedRunningTime="2026-09-16 17:45:49.144553166 +0000 UTC m=+1289.136648762"
Sep 16 17:45:50.124569 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:50.124531    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:45:55.086132 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:55.086088    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:45:55.128948 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:55.128916    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:45:55.129579 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:45:55.129543    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:05.085667 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:05.085621    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:46:05.129639 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:05.129601    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:15.086472 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:15.086430    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:46:15.129641 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:15.129597    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:25.086355 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:25.086312    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:46:25.129655 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:25.129609    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:35.086508 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:35.086468    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:46:35.129990 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:35.129955    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:45.086170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:45.086065    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:46:45.129672 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:45.129636    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:46:55.086946 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:55.086914    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:46:55.129519 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:46:55.129483    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:02.517269 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:02.517236    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:47:02.517687 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:02.517464    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" containerID="cri-o://178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0" gracePeriod=30
Sep 16 17:47:02.517687 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:02.517504    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kube-rbac-proxy" containerID="cri-o://a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49" gracePeriod=30
Sep 16 17:47:02.575117 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:02.575076    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:47:03.391809 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:03.391775    2542 generic.go:356] "Generic (PLEG): container finished" podID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerID="a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49" exitCode=2
Sep 16 17:47:03.391992 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:03.391837    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerDied","Data":"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"}
Sep 16 17:47:05.081271 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:05.081230    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.22:8643/healthz\": dial tcp 10.132.0.22:8643: connect: connection refused"
Sep 16 17:47:05.086564 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:05.086541    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.22:8080: connect: connection refused"
Sep 16 17:47:05.130782 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:05.130745    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:07.614031 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.613990    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:47:07.709949 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.709913    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:47:07.853187 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.853162    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:47:07.965115 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965081    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert\") pod \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") "
Sep 16 17:47:07.965292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965131    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2g6z\" (UniqueName: \"kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z\") pod \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") "
Sep 16 17:47:07.965292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965153    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls\") pod \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") "
Sep 16 17:47:07.965292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965191    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config\") pod \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") "
Sep 16 17:47:07.965292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965209    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location\") pod \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\" (UID: \"de03e6cd-f7f2-4ee6-9f1d-95a188eac476\") "
Sep 16 17:47:07.965634 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965602    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "de03e6cd-f7f2-4ee6-9f1d-95a188eac476" (UID: "de03e6cd-f7f2-4ee6-9f1d-95a188eac476"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:47:07.965634 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965607    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "de03e6cd-f7f2-4ee6-9f1d-95a188eac476" (UID: "de03e6cd-f7f2-4ee6-9f1d-95a188eac476"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:47:07.965826 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.965606    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "de03e6cd-f7f2-4ee6-9f1d-95a188eac476" (UID: "de03e6cd-f7f2-4ee6-9f1d-95a188eac476"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:47:07.967300 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.967274    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "de03e6cd-f7f2-4ee6-9f1d-95a188eac476" (UID: "de03e6cd-f7f2-4ee6-9f1d-95a188eac476"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:47:07.967401 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:07.967336    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z" (OuterVolumeSpecName: "kube-api-access-v2g6z") pod "de03e6cd-f7f2-4ee6-9f1d-95a188eac476" (UID: "de03e6cd-f7f2-4ee6-9f1d-95a188eac476"). InnerVolumeSpecName "kube-api-access-v2g6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:47:08.066161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.066118    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2g6z\" (UniqueName: \"kubernetes.io/projected/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-api-access-v2g6z\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:47:08.066161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.066155    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:47:08.066161 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.066170    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:47:08.066410 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.066181    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:47:08.066410 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.066192    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de03e6cd-f7f2-4ee6-9f1d-95a188eac476-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:47:08.410795 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.410756    2542 generic.go:356] "Generic (PLEG): container finished" podID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerID="178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0" exitCode=0
Sep 16 17:47:08.410983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.410818    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"
Sep 16 17:47:08.410983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.410836    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerDied","Data":"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"}
Sep 16 17:47:08.410983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.410946    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn" event={"ID":"de03e6cd-f7f2-4ee6-9f1d-95a188eac476","Type":"ContainerDied","Data":"979a51c23f4a254b461a829e3850e02c31d8cdcf226b73d7d7c934cc0c277b15"}
Sep 16 17:47:08.410983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.410974    2542 scope.go:117] "RemoveContainer" containerID="a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"
Sep 16 17:47:08.419050 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.419034    2542 scope.go:117] "RemoveContainer" containerID="178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"
Sep 16 17:47:08.428093 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.428075    2542 scope.go:117] "RemoveContainer" containerID="dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"
Sep 16 17:47:08.435132 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.435109    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:47:08.435678 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.435665    2542 scope.go:117] "RemoveContainer" containerID="a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"
Sep 16 17:47:08.435971 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:47:08.435955    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49\": container with ID starting with a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49 not found: ID does not exist" containerID="a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"
Sep 16 17:47:08.436027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.435979    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49"} err="failed to get container status \"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49\": rpc error: code = NotFound desc = could not find container \"a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49\": container with ID starting with a8145f5aaf3f41abae2bd4fa37876f89703726c8e7b2889a7fe6319dacd0ca49 not found: ID does not exist"
Sep 16 17:47:08.436027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.435995    2542 scope.go:117] "RemoveContainer" containerID="178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"
Sep 16 17:47:08.436233 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:47:08.436205    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0\": container with ID starting with 178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0 not found: ID does not exist" containerID="178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"
Sep 16 17:47:08.436278 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.436237    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0"} err="failed to get container status \"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0\": rpc error: code = NotFound desc = could not find container \"178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0\": container with ID starting with 178aae460c53029c003c1f526b801a0f9af7685b4ef1bdd16213b98159a977e0 not found: ID does not exist"
Sep 16 17:47:08.436278 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.436255    2542 scope.go:117] "RemoveContainer" containerID="dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"
Sep 16 17:47:08.436478 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:47:08.436463    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5\": container with ID starting with dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5 not found: ID does not exist" containerID="dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"
Sep 16 17:47:08.436516 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.436482    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5"} err="failed to get container status \"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5\": rpc error: code = NotFound desc = could not find container \"dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5\": container with ID starting with dca693da017fa8b7686f47203965fdcb949a4116513c2a7fbc2c3b435db2c2d5 not found: ID does not exist"
Sep 16 17:47:08.439457 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.439439    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/raw-sklearn-7c42d-predictor-657dfb5546-2pxvn"]
Sep 16 17:47:08.622472 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:08.622439    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" path="/var/lib/kubelet/pods/de03e6cd-f7f2-4ee6-9f1d-95a188eac476/volumes"
Sep 16 17:47:09.618146 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:09.618109    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:19.618587 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:19.618548    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:29.618746 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:29.618700    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:39.618368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:39.618322    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:49.618016 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:49.617972    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:47:53.798599 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:53.798563    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:47:59.618545 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:47:59.618495    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:48:09.618466 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:09.618421    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:48:19.618527 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:19.618440    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:48:22.618295 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:22.618247    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:48:24.227611 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:24.227571    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:32.621863 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:32.621821    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:48:38.015646 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:38.015588    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:42.425770 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.425729    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:48:42.426276 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.426064    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" containerID="cri-o://78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2" gracePeriod=30
Sep 16 17:48:42.426276 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.426089    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy" containerID="cri-o://1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb" gracePeriod=30
Sep 16 17:48:42.439263 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.439236    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:42.619027 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.618988    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: connect: connection refused"
Sep 16 17:48:42.742453 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.742363    2542 generic.go:356] "Generic (PLEG): container finished" podID="96331fdb-8031-4d57-b1f6-8bde90653823" containerID="1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb" exitCode=2
Sep 16 17:48:42.742453 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.742402    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerDied","Data":"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"}
Sep 16 17:48:42.972090 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:42.972040    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:43.194490 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:43.194453    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:43.314579 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:43.314547    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:45.125540 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:45.125489    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.23:8643/healthz\": dial tcp 10.132.0.23:8643: connect: connection refused"
Sep 16 17:48:49.940804 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:49.940765    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:50.079970 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:50.079930    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:48:50.125018 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:50.124976    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.23:8643/healthz\": dial tcp 10.132.0.23:8643: connect: connection refused"
Sep 16 17:48:52.666602 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.666578    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:48:52.777906 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.777797    2542 generic.go:356] "Generic (PLEG): container finished" podID="96331fdb-8031-4d57-b1f6-8bde90653823" containerID="78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2" exitCode=0
Sep 16 17:48:52.777906 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.777843    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerDied","Data":"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"}
Sep 16 17:48:52.777906 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.777878    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"
Sep 16 17:48:52.778107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.777929    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" event={"ID":"96331fdb-8031-4d57-b1f6-8bde90653823","Type":"ContainerDied","Data":"c5d0418f5c0cca54a9287550e3102d1ef23c1d3651076e1c0e87286b7178e693"}
Sep 16 17:48:52.778107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.777968    2542 scope.go:117] "RemoveContainer" containerID="1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"
Sep 16 17:48:52.785509 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.785488    2542 scope.go:117] "RemoveContainer" containerID="78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"
Sep 16 17:48:52.792340 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.792319    2542 scope.go:117] "RemoveContainer" containerID="81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"
Sep 16 17:48:52.799067 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.799051    2542 scope.go:117] "RemoveContainer" containerID="1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"
Sep 16 17:48:52.799367 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:48:52.799347    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb\": container with ID starting with 1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb not found: ID does not exist" containerID="1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"
Sep 16 17:48:52.799441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.799375    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb"} err="failed to get container status \"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb\": rpc error: code = NotFound desc = could not find container \"1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb\": container with ID starting with 1daf102d2677b0df475197c833861cff507b525fb6615cfaf0c3faf899e09ebb not found: ID does not exist"
Sep 16 17:48:52.799441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.799392    2542 scope.go:117] "RemoveContainer" containerID="78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"
Sep 16 17:48:52.799646 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:48:52.799631    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2\": container with ID starting with 78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2 not found: ID does not exist" containerID="78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"
Sep 16 17:48:52.799695 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.799647    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2"} err="failed to get container status \"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2\": rpc error: code = NotFound desc = could not find container \"78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2\": container with ID starting with 78e31bab0e0677ff5e63be0c3941157b0cff46468fa6795796c2bd0fcb0ad7f2 not found: ID does not exist"
Sep 16 17:48:52.799695 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.799661    2542 scope.go:117] "RemoveContainer" containerID="81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"
Sep 16 17:48:52.800193 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:48:52.800176    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf\": container with ID starting with 81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf not found: ID does not exist" containerID="81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"
Sep 16 17:48:52.800254 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.800197    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf"} err="failed to get container status \"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf\": rpc error: code = NotFound desc = could not find container \"81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf\": container with ID starting with 81b3cb5e3d8aeffa685dd05fbe2b0e65fbbe33bd31a0c6f28560ffa6889452cf not found: ID does not exist"
Sep 16 17:48:52.804456 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804437    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") pod \"96331fdb-8031-4d57-b1f6-8bde90653823\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") "
Sep 16 17:48:52.804513 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804485    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert\") pod \"96331fdb-8031-4d57-b1f6-8bde90653823\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") "
Sep 16 17:48:52.804555 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804513    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config\") pod \"96331fdb-8031-4d57-b1f6-8bde90653823\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") "
Sep 16 17:48:52.804630 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804616    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzjgf\" (UniqueName: \"kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf\") pod \"96331fdb-8031-4d57-b1f6-8bde90653823\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") "
Sep 16 17:48:52.804666 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804652    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location\") pod \"96331fdb-8031-4d57-b1f6-8bde90653823\" (UID: \"96331fdb-8031-4d57-b1f6-8bde90653823\") "
Sep 16 17:48:52.804838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804815    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "96331fdb-8031-4d57-b1f6-8bde90653823" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:48:52.804838 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.804819    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "96331fdb-8031-4d57-b1f6-8bde90653823" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:48:52.805046 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.805019    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "96331fdb-8031-4d57-b1f6-8bde90653823" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:48:52.806385 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.806365    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "96331fdb-8031-4d57-b1f6-8bde90653823" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:48:52.806484 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.806464    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf" (OuterVolumeSpecName: "kube-api-access-xzjgf") pod "96331fdb-8031-4d57-b1f6-8bde90653823" (UID: "96331fdb-8031-4d57-b1f6-8bde90653823"). InnerVolumeSpecName "kube-api-access-xzjgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:48:52.905959 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.905919    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/96331fdb-8031-4d57-b1f6-8bde90653823-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:48:52.905959 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.905950    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:48:52.905959 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.905960    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/96331fdb-8031-4d57-b1f6-8bde90653823-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:48:52.905959 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.905969    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzjgf\" (UniqueName: \"kubernetes.io/projected/96331fdb-8031-4d57-b1f6-8bde90653823-kube-api-access-xzjgf\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:48:52.906223 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:52.905977    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/96331fdb-8031-4d57-b1f6-8bde90653823-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:48:53.106368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:53.106333    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:48:53.117669 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:53.117640    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw"]
Sep 16 17:48:53.618015 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:53.617971    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-scale-raw-be77d-predictor-7cffc4447b-mf5hw" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.23:8080: i/o timeout"
Sep 16 17:48:54.621974 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:48:54.621942    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" path="/var/lib/kubelet/pods/96331fdb-8031-4d57-b1f6-8bde90653823/volumes"
Sep 16 17:49:20.508493 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:49:20.508464    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:49:20.510608 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:49:20.510590    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:50:13.176478 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:13.176444    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:13.270180 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:13.270141    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:18.372268 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.372229    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:18.478927 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.478894    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:50:18.479188 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479175    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container"
Sep 16 17:50:18.479231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479192    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container"
Sep 16 17:50:18.479231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479207    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="storage-initializer"
Sep 16 17:50:18.479231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479212    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="storage-initializer"
Sep 16 17:50:18.479231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479221    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kube-rbac-proxy"
Sep 16 17:50:18.479231 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479227    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kube-rbac-proxy"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479235    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479240    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479248    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="storage-initializer"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479252    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="storage-initializer"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479258    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479264    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479309    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kube-rbac-proxy"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479316    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kserve-container"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479322    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="de03e6cd-f7f2-4ee6-9f1d-95a188eac476" containerName="kserve-container"
Sep 16 17:50:18.479380 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.479327    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="96331fdb-8031-4d57-b1f6-8bde90653823" containerName="kube-rbac-proxy"
Sep 16 17:50:18.482286 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.482270    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.486036 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.486015    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-9bc9t\""
Sep 16 17:50:18.486176 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.486161    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\""
Sep 16 17:50:18.487178 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.487166    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Sep 16 17:50:18.487601 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.487584    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Sep 16 17:50:18.494104 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.494088    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 16 17:50:18.494632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.494618    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\""
Sep 16 17:50:18.496218 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.496203    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 16 17:50:18.507104 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.507080    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:50:18.617689 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.617618    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.617979 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.617708    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.617979 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.617743    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bvz\" (UniqueName: \"kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.617979 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.617824    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.617979 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.617900    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.620330 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.620279    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:18.718753 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.718713    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.718983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.718776    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.718983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.718805    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.718983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.718829    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bvz\" (UniqueName: \"kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.718983 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.718916    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.719366 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.719341    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.719542 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.719520    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.719606 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.719563    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.721337 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.721320    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.728224 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.728194    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bvz\" (UniqueName: \"kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz\") pod \"isvc-canary-promote-v2-predictor-b8964cf76-qb8fd\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.749839 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.749795    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:18.792211 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.792175    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:18.945991 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:18.945959    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:50:19.073517 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:19.073492    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:50:19.074935 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:50:19.074903    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fa9b9e_cfe6_433c_8863_dff2ca72ddc0.slice/crio-1a0d2991e1e2022df368be2f637f987c5a7959ceb3a18708f42e47ac59cefeab WatchSource:0}: Error finding container 1a0d2991e1e2022df368be2f637f987c5a7959ceb3a18708f42e47ac59cefeab: Status 404 returned error can't find the container with id 1a0d2991e1e2022df368be2f637f987c5a7959ceb3a18708f42e47ac59cefeab
Sep 16 17:50:20.070188 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:20.070147    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerStarted","Data":"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"}
Sep 16 17:50:20.070602 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:20.070208    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerStarted","Data":"1a0d2991e1e2022df368be2f637f987c5a7959ceb3a18708f42e47ac59cefeab"}
Sep 16 17:50:24.085899 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:24.085829    2542 generic.go:356] "Generic (PLEG): container finished" podID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerID="dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb" exitCode=0
Sep 16 17:50:24.086330 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:24.085906    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerDied","Data":"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"}
Sep 16 17:50:25.092796 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.092753    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerStarted","Data":"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"}
Sep 16 17:50:25.093309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.092816    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerStarted","Data":"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"}
Sep 16 17:50:25.093309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.093105    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:25.093309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.093236    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:25.094537 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.094510    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:50:25.117236 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:25.117175    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podStartSLOduration=3.170677723 podStartE2EDuration="7.117160091s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.946482368s" isStatefulPod=false podCreationTimestamp="2026-09-16 17:50:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:50:25.116197579 +0000 UTC m=+1565.108293173" watchObservedRunningTime="2026-09-16 17:50:25.117160091 +0000 UTC m=+1565.109255685"
Sep 16 17:50:26.096127 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:26.096072    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:50:31.101693 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:31.101663    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:50:31.102170 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:31.102147    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:50:41.102160 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:41.102117    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:50:51.102831 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:50:51.102784    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:51:01.102923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:01.102878    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:51:11.103093 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:11.103050    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:51:21.102623 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:21.102538    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:51:31.102122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:31.102080    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" probeResult="failure" output="dial tcp 10.132.0.24:8080: connect: connection refused"
Sep 16 17:51:37.917412 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:37.917378    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:51:38.622334 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:38.622306    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:51:48.363201 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:48.363162    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:51:51.759484 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:51:51.759430    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:14.727816 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:14.727779    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:18.635678 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:18.635647    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:53:18.636123 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:18.635898    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container" containerID="cri-o://620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127" gracePeriod=30
Sep 16 17:53:18.636123 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:18.635948    2542 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kube-rbac-proxy" containerID="cri-o://ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3" gracePeriod=30
Sep 16 17:53:18.683273 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:18.683237    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:19.709774 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:19.709738    2542 generic.go:356] "Generic (PLEG): container finished" podID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerID="ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3" exitCode=2
Sep 16 17:53:19.710182 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:19.709808    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerDied","Data":"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"}
Sep 16 17:53:21.096923 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:21.096879    2542 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.132.0.24:8643/healthz\": dial tcp 10.132.0.24:8643: connect: connection refused"
Sep 16 17:53:23.569833 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.569802    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:53:23.702692 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702599    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location\") pod \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") "
Sep 16 17:53:23.702692 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702648    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls\") pod \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") "
Sep 16 17:53:23.702692 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702671    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6bvz\" (UniqueName: \"kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz\") pod \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") "
Sep 16 17:53:23.703017 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702700    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config\") pod \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") "
Sep 16 17:53:23.703017 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702728    2542 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert\") pod \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\" (UID: \"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0\") "
Sep 16 17:53:23.703120 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.702971    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" (UID: "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 17:53:23.703176 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.703123    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" (UID: "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:53:23.703176 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.703146    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" (UID: "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 17:53:23.704760 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.704740    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" (UID: "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 17:53:23.704863 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.704832    2542 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz" (OuterVolumeSpecName: "kube-api-access-z6bvz") pod "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" (UID: "76fa9b9e-cfe6-433c-8863-dff2ca72ddc0"). InnerVolumeSpecName "kube-api-access-z6bvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 17:53:23.727284 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.727252    2542 generic.go:356] "Generic (PLEG): container finished" podID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerID="620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127" exitCode=0
Sep 16 17:53:23.727409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.727303    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerDied","Data":"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"}
Sep 16 17:53:23.727409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.727316    2542 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"
Sep 16 17:53:23.727409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.727351    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd" event={"ID":"76fa9b9e-cfe6-433c-8863-dff2ca72ddc0","Type":"ContainerDied","Data":"1a0d2991e1e2022df368be2f637f987c5a7959ceb3a18708f42e47ac59cefeab"}
Sep 16 17:53:23.727409 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.727377    2542 scope.go:117] "RemoveContainer" containerID="ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"
Sep 16 17:53:23.735718 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.735696    2542 scope.go:117] "RemoveContainer" containerID="620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"
Sep 16 17:53:23.742690 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.742673    2542 scope.go:117] "RemoveContainer" containerID="dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"
Sep 16 17:53:23.749777 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.749754    2542 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:53:23.750016 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750002    2542 scope.go:117] "RemoveContainer" containerID="ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"
Sep 16 17:53:23.750299 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:53:23.750259    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3\": container with ID starting with ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3 not found: ID does not exist" containerID="ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"
Sep 16 17:53:23.750370 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750294    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3"} err="failed to get container status \"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3\": rpc error: code = NotFound desc = could not find container \"ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3\": container with ID starting with ecb93257c1e5378e1ca9930b2ddedb831abb95121ead924f955a7d746ab67de3 not found: ID does not exist"
Sep 16 17:53:23.750370 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750318    2542 scope.go:117] "RemoveContainer" containerID="620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"
Sep 16 17:53:23.750550 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:53:23.750535    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127\": container with ID starting with 620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127 not found: ID does not exist" containerID="620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"
Sep 16 17:53:23.750589 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750556    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127"} err="failed to get container status \"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127\": rpc error: code = NotFound desc = could not find container \"620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127\": container with ID starting with 620f3aa2242cdede0715b65a7ccaa8f1db8a4c62c7ce2ec764b69e4f221d2127 not found: ID does not exist"
Sep 16 17:53:23.750589 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750570    2542 scope.go:117] "RemoveContainer" containerID="dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"
Sep 16 17:53:23.750779 ip-10-0-134-96 kubenswrapper[2542]: E0916 17:53:23.750761    2542 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb\": container with ID starting with dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb not found: ID does not exist" containerID="dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"
Sep 16 17:53:23.750821 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.750783    2542 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb"} err="failed to get container status \"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb\": rpc error: code = NotFound desc = could not find container \"dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb\": container with ID starting with dbaae2f5bf5786b054dd6d0565bef4c7fb85a0a2fda574a645bdf0820c1969fb not found: ID does not exist"
Sep 16 17:53:23.755221 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.755200    2542 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-b8964cf76-qb8fd"]
Sep 16 17:53:23.803511 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.803482    2542 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kserve-provision-location\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:53:23.803632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.803514    2542 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-proxy-tls\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:53:23.803632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.803529    2542 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6bvz\" (UniqueName: \"kubernetes.io/projected/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-api-access-z6bvz\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:53:23.803632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.803538    2542 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-kube-rbac-proxy-sar-config\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:53:23.803632 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:23.803548    2542 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0-cabundle-cert\") on node \"ip-10-0-134-96.ec2.internal\" DevicePath \"\""
Sep 16 17:53:24.622309 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:24.622275    2542 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" path="/var/lib/kubelet/pods/76fa9b9e-cfe6-433c-8863-dff2ca72ddc0/volumes"
Sep 16 17:53:32.694121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:32.694089    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:33.840640 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:33.840606    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:37.655547 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:37.655512    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:42.716107 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:42.716070    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:53:43.164371 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:53:43.164338    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:54:20.529471 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:54:20.529422    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:54:20.531915 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:54:20.531893    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:55:18.266292 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:18.266257    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:18.335379 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:18.335345    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:19.067559 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:19.067525    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:19.219326 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:19.219291    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:50.528901 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:50.528797    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:50.727122 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:50.727089    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:50.843901 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:50.843800    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:53.722042 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:53.722009    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7knvj_34a69a06-bfeb-4602-a1d9-dbd5181b884f/global-pull-secret-syncer/0.log"
Sep 16 17:55:53.918345 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:53.918315    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log"
Sep 16 17:55:54.128475 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:54.128392    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-96.ec2.internal_d95d3581c8d80d714714e1182983c2fc/haproxy/0.log"
Sep 16 17:55:56.582722 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:56.582687    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:56.681832 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:56.681800    2542 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-722wm_63b6d3a8-e251-437b-95b6-853963dbe982/konnectivity-agent/0.log" limit=1024
Sep 16 17:55:58.353966 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:58.353938    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dd2c_c0b11b96-936e-48e4-8678-02aa0e249d47/node-exporter/0.log"
Sep 16 17:55:58.500748 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:58.500723    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dd2c_c0b11b96-936e-48e4-8678-02aa0e249d47/kube-rbac-proxy/0.log"
Sep 16 17:55:58.567225 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:55:58.567201    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2dd2c_c0b11b96-936e-48e4-8678-02aa0e249d47/init-textfile/0.log"
Sep 16 17:56:01.624984 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.624948    2542 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"]
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625188    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625198    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625214    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="storage-initializer"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625220    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="storage-initializer"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625233    2542 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kube-rbac-proxy"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625239    2542 state_mem.go:107] "Deleted CPUSet assignment" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kube-rbac-proxy"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625276    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kserve-container"
Sep 16 17:56:01.625368 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.625284    2542 memory_manager.go:368] "RemoveStaleState removing state" podUID="76fa9b9e-cfe6-433c-8863-dff2ca72ddc0" containerName="kube-rbac-proxy"
Sep 16 17:56:01.628018 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.628001    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.635501 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.635479    2542 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-znfk5\"/\"default-dockercfg-bh4xb\""
Sep 16 17:56:01.636419 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.636388    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znfk5\"/\"openshift-service-ca.crt\""
Sep 16 17:56:01.636531 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.636441    2542 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znfk5\"/\"kube-root-ca.crt\""
Sep 16 17:56:01.639034 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.639012    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"]
Sep 16 17:56:01.711232 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.711197    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-proc\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.711441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.711239    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hfdg\" (UniqueName: \"kubernetes.io/projected/17e7b82b-dcb8-4dce-827e-dae0062dc173-kube-api-access-6hfdg\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.711441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.711307    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-sys\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.711441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.711344    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-podres\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.711441 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.711370    2542 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-lib-modules\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812225 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812188    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-sys\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812225 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812224    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-podres\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812248    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-lib-modules\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812296    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-proc\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812322    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-sys\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812333    2542 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hfdg\" (UniqueName: \"kubernetes.io/projected/17e7b82b-dcb8-4dce-827e-dae0062dc173-kube-api-access-6hfdg\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812370    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-podres\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812378    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-proc\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.812431 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.812428    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/17e7b82b-dcb8-4dce-827e-dae0062dc173-lib-modules\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.821560 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.821526    2542 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hfdg\" (UniqueName: \"kubernetes.io/projected/17e7b82b-dcb8-4dce-827e-dae0062dc173-kube-api-access-6hfdg\") pod \"perf-node-gather-daemonset-zmnr5\" (UID: \"17e7b82b-dcb8-4dce-827e-dae0062dc173\") " pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:01.936985 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:01.936950    2542 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:02.208713 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:02.208684    2542 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"]
Sep 16 17:56:02.211109 ip-10-0-134-96 kubenswrapper[2542]: W0916 17:56:02.211077    2542 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod17e7b82b_dcb8_4dce_827e_dae0062dc173.slice/crio-99f3769ed2a013362b7166ae10a83f77f985dd7ef5e3d8fa2051b25e68a98e51 WatchSource:0}: Error finding container 99f3769ed2a013362b7166ae10a83f77f985dd7ef5e3d8fa2051b25e68a98e51: Status 404 returned error can't find the container with id 99f3769ed2a013362b7166ae10a83f77f985dd7ef5e3d8fa2051b25e68a98e51
Sep 16 17:56:02.262012 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:02.261975    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5" event={"ID":"17e7b82b-dcb8-4dce-827e-dae0062dc173","Type":"ContainerStarted","Data":"99f3769ed2a013362b7166ae10a83f77f985dd7ef5e3d8fa2051b25e68a98e51"}
Sep 16 17:56:03.271281 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:03.271239    2542 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5" event={"ID":"17e7b82b-dcb8-4dce-827e-dae0062dc173","Type":"ContainerStarted","Data":"8ddf3238dd47d9baa82a0243c489b65b3fa658914e05039f99b4eb09f5f6651e"}
Sep 16 17:56:03.271716 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:03.271350    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:03.293376 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:03.293322    2542 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5" podStartSLOduration=2.29330614 podStartE2EDuration="2.29330614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 17:56:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 17:56:03.292184967 +0000 UTC m=+1903.284280563" watchObservedRunningTime="2026-09-16 17:56:03.29330614 +0000 UTC m=+1903.285401736"
Sep 16 17:56:04.036121 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:04.036091    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rkb45_d92892f1-d976-4c50-ab9d-0398376fb6a1/dns/0.log"
Sep 16 17:56:04.083528 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:04.083502    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rkb45_d92892f1-d976-4c50-ab9d-0398376fb6a1/kube-rbac-proxy/0.log"
Sep 16 17:56:04.197019 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:04.196995    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-czk72_49da429c-403a-46cf-a214-c584c906f98f/dns-node-resolver/0.log"
Sep 16 17:56:04.885297 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:04.885271    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-28mvz_74e282fd-2c09-4e91-af47-b93ffce7077c/node-ca/0.log"
Sep 16 17:56:06.532525 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:06.532499    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5d5hd_60d86da6-1f33-439b-9ee5-a289ed6c00e8/serve-healthcheck-canary/0.log"
Sep 16 17:56:07.021162 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:07.021136    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmhdq_1713059b-bf0c-46fd-9b41-2a70816d3645/kube-rbac-proxy/0.log"
Sep 16 17:56:07.048864 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:07.048837    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmhdq_1713059b-bf0c-46fd-9b41-2a70816d3645/exporter/0.log"
Sep 16 17:56:07.075172 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:07.075144    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pmhdq_1713059b-bf0c-46fd-9b41-2a70816d3645/extractor/0.log"
Sep 16 17:56:09.284991 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:09.284965    2542 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-znfk5/perf-node-gather-daemonset-zmnr5"
Sep 16 17:56:09.665304 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:09.665278    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-7qmg4_ed012f33-ce75-46f2-8e2c-d45ae935a42e/manager/0.log"
Sep 16 17:56:16.552061 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.552029    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7z6nl_49353fb8-5cda-4fc2-9660-8584d100b573/kube-multus/0.log"
Sep 16 17:56:16.857741 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.857668    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/kube-multus-additional-cni-plugins/0.log"
Sep 16 17:56:16.893877 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.893835    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/egress-router-binary-copy/0.log"
Sep 16 17:56:16.929700 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.929670    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/cni-plugins/0.log"
Sep 16 17:56:16.956084 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.956057    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/bond-cni-plugin/0.log"
Sep 16 17:56:16.995355 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:16.995330    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/routeoverride-cni/0.log"
Sep 16 17:56:17.027233 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:17.027208    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/whereabouts-cni-bincopy/0.log"
Sep 16 17:56:17.083770 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:17.083739    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ddjl5_65383bc0-ebcb-4a48-87a1-fcaa1855f17e/whereabouts-cni/0.log"
Sep 16 17:56:17.626451 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:17.626422    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s76cv_f313bc9f-a278-4e04-aaef-e2e9f30c18db/network-metrics-daemon/0.log"
Sep 16 17:56:17.671349 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:17.671320    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s76cv_f313bc9f-a278-4e04-aaef-e2e9f30c18db/kube-rbac-proxy/0.log"
Sep 16 17:56:18.831773 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:18.831745    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-controller/0.log"
Sep 16 17:56:18.870320 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:18.870291    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/0.log"
Sep 16 17:56:18.871981 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:18.871959    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/ovn-acl-logging/1.log"
Sep 16 17:56:18.904804 ip-10-0-134-96 kubenswrapper[2542]: I0916 17:56:18.904773    2542 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mrjcs_dd660502-1d96-4943-b847-09265cc97f7f/kube-rbac-proxy-node/0.log"