Sep 17 20:47:12.644928 ip-10-0-138-134 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 17 20:47:12.644937 ip-10-0-138-134 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 17 20:47:12.644944 ip-10-0-138-134 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 17 20:47:12.645087 ip-10-0-138-134 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 17 20:47:22.742552 ip-10-0-138-134 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 17 20:47:22.742572 ip-10-0-138-134 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 0ff412e777644664a72ed78303ed4229 --
Sep 17 20:49:19.006726 ip-10-0-138-134 systemd[1]: Starting Kubernetes Kubelet...
Sep 17 20:49:19.336386 ip-10-0-138-134 kubenswrapper[2587]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:49:19.336386 ip-10-0-138-134 kubenswrapper[2587]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 17 20:49:19.336386 ip-10-0-138-134 kubenswrapper[2587]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:49:19.336386 ip-10-0-138-134 kubenswrapper[2587]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 17 20:49:19.336386 ip-10-0-138-134 kubenswrapper[2587]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 17 20:49:19.337287 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.337176    2587 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 17 20:49:19.340219 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340195    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.340219 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340212    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.340219 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340218    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.340219 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340223    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.340219 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340227    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340231    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340236    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340240    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340243    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340247    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340251    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340255    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340258    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340262    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340266    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340270    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340274    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340278    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340282    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340286    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340290    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340294    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340299    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340303    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.340533 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340307    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340312    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340316    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340329    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340333    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340336    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340340    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340344    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340349    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340353    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340357    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340361    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340366    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340370    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340375    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340379    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340384    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340388    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340392    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340396    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.341285 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340401    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340405    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340411    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340415    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340435    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340440    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340445    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340448    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340453    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340458    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340463    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340470    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340477    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340482    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340487    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340491    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340496    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340500    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340505    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340510    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.341812 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340517    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340521    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340527    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340532    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340536    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340542    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340547    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340551    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340556    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340560    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340564    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340569    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340573    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340577    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340581    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340585    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340591    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340595    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340599    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340603    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.342315 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340607    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340611    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340617    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340622    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340627    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340631    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340636    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340640    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340644    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340649    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340654    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340659    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340663    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340667    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.340671    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341628    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341640    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341643    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341647    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.342838 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341650    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341652    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341655    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341658    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341660    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341663    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341665    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341668    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341671    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341673    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341676    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341678    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341681    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341683    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341687    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341689    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341692    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341694    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341697    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341699    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341701    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.343299 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341704    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341706    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341709    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341712    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341714    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341717    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341720    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341723    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341727    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341730    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341732    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341736    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341738    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341741    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341744    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341746    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341749    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341752    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341754    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341757    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.343836 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341760    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341762    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341765    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341768    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341770    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341773    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341776    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341779    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341781    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341784    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341786    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341789    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341791    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341794    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341796    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341799    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341801    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341804    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341806    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341809    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.344337 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341812    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341815    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341818    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341821    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341823    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341826    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341828    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341831    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341834    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341836    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341839    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341844    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341848    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341852    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341856    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341859    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341862    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341866    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.344848 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341869    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341872    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341874    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341877    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341880    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341882    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341885    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341888    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341890    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341893    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341895    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341898    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341900    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341903    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341907    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.341909    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342272    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342277    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342279    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.345279 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342283    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342285    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342288    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342291    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342293    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342296    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342299    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342301    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342304    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342306    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342309    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342311    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342314    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342317    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342319    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342322    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342325    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342327    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342330    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342333    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.345775 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342335    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342338    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342341    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342344    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342347    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342351    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342354    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342358    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342361    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342364    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342367    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342370    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342372    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342375    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342377    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342379    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342382    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342384    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342387    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.346255 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342390    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342392    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342395    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342399    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342402    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342405    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342408    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342411    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342414    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342417    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342434    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342438    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342440    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342443    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342446    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342448    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342451    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342454    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342457    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.346747 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342460    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342463    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342466    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342468    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342471    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342474    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342478    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342481    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342484    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342486    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342489    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342491    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342494    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342496    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342499    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342501    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342504    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342507    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342509    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342512    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.347242 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342514    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342517    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342519    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342522    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342525    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342528    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342530    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342533    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342535    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342538    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342541    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342543    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342546    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342549    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342551    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342554    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342556    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.342559    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342632    2587 flags.go:64] FLAG: --address="0.0.0.0"
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342669    2587 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342677    2587 flags.go:64] FLAG: --anonymous-auth="true"
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342681    2587 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 17 20:49:19.347736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342685    2587 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342689    2587 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342693    2587 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342698    2587 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342701    2587 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342704    2587 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342708    2587 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342711    2587 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342714    2587 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342717    2587 flags.go:64] FLAG: --cgroup-root=""
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342720    2587 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342723    2587 flags.go:64] FLAG: --client-ca-file=""
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342726    2587 flags.go:64] FLAG: --cloud-provider="external"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342729    2587 flags.go:64] FLAG: --cluster-dns="[]"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342733    2587 flags.go:64] FLAG: --cluster-domain=""
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342736    2587 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342739    2587 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342742    2587 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342746    2587 flags.go:64] FLAG: --container-log-max-files="5"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342749    2587 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342752    2587 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342755    2587 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342759    2587 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342762    2587 flags.go:64] FLAG: --contention-profiling="false"
Sep 17 20:49:19.348270 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342764    2587 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342767    2587 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342770    2587 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342773    2587 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342778    2587 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342781    2587 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342784    2587 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342787    2587 flags.go:64] FLAG: --enable-load-reader="false"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342790    2587 flags.go:64] FLAG: --enable-server="true"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342793    2587 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342801    2587 flags.go:64] FLAG: --event-burst="100"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342804    2587 flags.go:64] FLAG: --event-qps="50"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342807    2587 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342812    2587 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342815    2587 flags.go:64] FLAG: --eviction-hard=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342819    2587 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342822    2587 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342825    2587 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342828    2587 flags.go:64] FLAG: --eviction-soft=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342831    2587 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342833    2587 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342836    2587 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342839    2587 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342842    2587 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342845    2587 flags.go:64] FLAG: --fail-swap-on="true"
Sep 17 20:49:19.348851 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342850    2587 flags.go:64] FLAG: --feature-gates=""
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342854    2587 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342857    2587 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342860    2587 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342863    2587 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342866    2587 flags.go:64] FLAG: --healthz-port="10248"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342869    2587 flags.go:64] FLAG: --help="false"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342872    2587 flags.go:64] FLAG: --hostname-override="ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342875    2587 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342878    2587 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342881    2587 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342884    2587 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342891    2587 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342894    2587 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342896    2587 flags.go:64] FLAG: --image-service-endpoint=""
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342899    2587 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342902    2587 flags.go:64] FLAG: --kube-api-burst="100"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342905    2587 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342908    2587 flags.go:64] FLAG: --kube-api-qps="50"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342910    2587 flags.go:64] FLAG: --kube-reserved=""
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342915    2587 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342917    2587 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342920    2587 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342923    2587 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 17 20:49:19.349492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342926    2587 flags.go:64] FLAG: --lock-file=""
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342928    2587 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342932    2587 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342935    2587 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342940    2587 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342943    2587 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342945    2587 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342949    2587 flags.go:64] FLAG: --logging-format="text"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342955    2587 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342958    2587 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342961    2587 flags.go:64] FLAG: --manifest-url=""
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342964    2587 flags.go:64] FLAG: --manifest-url-header=""
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342968    2587 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342971    2587 flags.go:64] FLAG: --max-open-files="1000000"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342976    2587 flags.go:64] FLAG: --max-pods="110"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342979    2587 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342982    2587 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342985    2587 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342988    2587 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342990    2587 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342993    2587 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.342997    2587 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343004    2587 flags.go:64] FLAG: --node-status-max-images="50"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343007    2587 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343010    2587 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 17 20:49:19.350094 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343013    2587 flags.go:64] FLAG: --pod-cidr=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343016    2587 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343021    2587 flags.go:64] FLAG: --pod-manifest-path=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343025    2587 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343028    2587 flags.go:64] FLAG: --pods-per-core="0"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343031    2587 flags.go:64] FLAG: --port="10250"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343034    2587 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343037    2587 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d61fce9cd8d16cc9"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343040    2587 flags.go:64] FLAG: --qos-reserved=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343043    2587 flags.go:64] FLAG: --read-only-port="10255"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343046    2587 flags.go:64] FLAG: --register-node="true"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343049    2587 flags.go:64] FLAG: --register-with-taints=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343052    2587 flags.go:64] FLAG: --registry-burst="10"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343055    2587 flags.go:64] FLAG: --registry-qps="5"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343058    2587 flags.go:64] FLAG: --reserved-cpus=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343061    2587 flags.go:64] FLAG: --reserved-memory=""
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343066    2587 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343069    2587 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343071    2587 flags.go:64] FLAG: --rotate-certificates="false"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343074    2587 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343077    2587 flags.go:64] FLAG: --runonce="false"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343080    2587 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343083    2587 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343086    2587 flags.go:64] FLAG: --seccomp-default="false"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343089    2587 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 17 20:49:19.350752 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343091    2587 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343094    2587 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343097    2587 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343100    2587 flags.go:64] FLAG: --storage-driver-password="root"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343104    2587 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343107    2587 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343109    2587 flags.go:64] FLAG: --storage-driver-user="root"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343112    2587 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343115    2587 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343118    2587 flags.go:64] FLAG: --system-cgroups=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343122    2587 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343127    2587 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343130    2587 flags.go:64] FLAG: --tls-cert-file=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343132    2587 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343136    2587 flags.go:64] FLAG: --tls-min-version=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343139    2587 flags.go:64] FLAG: --tls-private-key-file=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343142    2587 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343145    2587 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343147    2587 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343150    2587 flags.go:64] FLAG: --v="2"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343154    2587 flags.go:64] FLAG: --version="false"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343158    2587 flags.go:64] FLAG: --vmodule=""
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343162    2587 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.343166    2587 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343278    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.351352 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343282    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343285    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343288    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343291    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343294    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343297    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343300    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343303    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343306    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343308    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343311    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343314    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343317    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343320    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343322    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343325    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343327    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343334    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343337    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.352030 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343340    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343343    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343346    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343348    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343351    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343353    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343356    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343358    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343361    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343364    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343366    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343370    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343373    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343376    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343379    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343382    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343384    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343387    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343390    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343392    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.352500 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343395    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343398    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343402    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343405    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343408    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343410    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343414    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343416    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343419    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343434    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343439    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343442    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343444    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343447    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343450    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343452    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343455    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343458    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343460    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343463    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.352982 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343465    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343468    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343471    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343475    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343478    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343481    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343483    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343486    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343489    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343491    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343495    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343497    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343500    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343502    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343505    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343507    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343510    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343512    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343515    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343518    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.353487 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343521    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343523    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343528    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343530    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343533    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343536    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343538    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343541    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343543    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343546    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343549    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343552    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343554    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343557    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343559    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343563    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343567    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343571    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.353971 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.343574    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.344090    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.351481    2587 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.351499    2587 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.352079    2587 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.352086    2587 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352163    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352168    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352172    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352175    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352177    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352180    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352184    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352192    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352195    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.354448 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352198    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352201    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352215    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352218    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352221    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352224    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352226    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352229    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352231    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352234    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352237    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352239    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352242    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352245    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352247    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352250    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352252    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352255    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352257    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352260    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.354817 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352263    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352266    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352269    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352272    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352275    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352277    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352280    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352282    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352285    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352288    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352290    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352298    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352303    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352306    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352310    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352313    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352316    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352318    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352322    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.355300 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352325    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352327    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352330    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352332    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352334    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352337    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352340    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352342    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352345    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352347    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352350    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352353    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352357    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352361    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352364    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352368    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352371    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352373    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352376    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.355827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352378    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352382    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352386    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352389    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352391    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352393    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352401    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352404    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352407    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352409    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352412    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352414    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352417    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352433    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352436    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352439    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352441    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352444    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352447    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.356303 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352449    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352452    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352454    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352457    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352459    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352462    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352465    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352468    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352470    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352473    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352476    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352478    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352481    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.352486    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352637    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 17 20:49:19.356769 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352642    2587 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352647    2587 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352650    2587 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352653    2587 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352656    2587 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352667    2587 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352669    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352672    2587 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352674    2587 feature_gate.go:328] unrecognized feature gate: Example2
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352677    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352680    2587 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352682    2587 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352685    2587 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352687    2587 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352690    2587 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352693    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352695    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352698    2587 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352700    2587 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352703    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 17 20:49:19.357155 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352705    2587 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352708    2587 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352711    2587 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352713    2587 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352717    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352720    2587 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352722    2587 feature_gate.go:328] unrecognized feature gate: Example
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352725    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352728    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352730    2587 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352733    2587 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352736    2587 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352739    2587 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352741    2587 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352744    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352746    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352749    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352751    2587 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352759    2587 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352762    2587 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 17 20:49:19.357650 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352765    2587 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352767    2587 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352769    2587 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352772    2587 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352776    2587 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352779    2587 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352782    2587 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352785    2587 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352788    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352791    2587 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352793    2587 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352796    2587 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352798    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352801    2587 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352803    2587 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352806    2587 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352808    2587 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352811    2587 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352814    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 17 20:49:19.358130 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352816    2587 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352819    2587 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352822    2587 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352824    2587 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352827    2587 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352830    2587 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352832    2587 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352835    2587 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352837    2587 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352840    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352842    2587 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352845    2587 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352853    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352856    2587 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352858    2587 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352861    2587 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352863    2587 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352866    2587 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352868    2587 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 17 20:49:19.358648 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352872    2587 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352875    2587 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352878    2587 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352881    2587 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352883    2587 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352886    2587 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352888    2587 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352891    2587 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352893    2587 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352896    2587 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352898    2587 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352901    2587 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352904    2587 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352906    2587 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352909    2587 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352911    2587 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352914    2587 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352916    2587 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352919    2587 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 17 20:49:19.359101 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:19.352921    2587 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.352927    2587 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.353067    2587 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.355195    2587 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.356225    2587 server.go:1019] "Starting client certificate rotation"
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.356323    2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.356355    2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 17 20:49:19.359721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.358389    2587 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 20:49:19.362686 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.362667    2587 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 17 20:49:19.373262 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.373243    2587 log.go:25] "Validated CRI v1 runtime API"
Sep 17 20:49:19.376718 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.376603    2587 log.go:25] "Validated CRI v1 image API"
Sep 17 20:49:19.377454 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.377441    2587 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 17 20:49:19.379988 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.379952    2587 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 925aee17-4e2b-4cb7-b343-32e2e83d943b:/dev/nvme0n1p4 a0bb884e-313b-47b5-848b-d80c342651f8:/dev/nvme0n1p3]
Sep 17 20:49:19.380080 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.379986    2587 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 17 20:49:19.384990 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.384872    2587 manager.go:216] Machine: {Timestamp:2026-09-17 20:49:19.383723126 +0000 UTC m=+0.280111211 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3142695 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27721e516a39d19151f28f8cf058aa SystemUUID:ec27721e-516a-39d1-9151-f28f8cf058aa BootID:0ff412e7-7764-4664-a72e-d78303ed4229 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f2:cf:f1:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f2:cf:f1:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:fd:70:a9:b9:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 17 20:49:19.384990 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.384980    2587 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 17 20:49:19.385138 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.385088    2587 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 17 20:49:19.387700 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.387676    2587 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 17 20:49:19.387885 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.387864    2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 20:49:19.387935 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.387702    2587 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-134.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 17 20:49:19.388458 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.388445    2587 topology_manager.go:138] "Creating topology manager with none policy"
Sep 17 20:49:19.388518 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.388464    2587 container_manager_linux.go:309] "Creating device plugin manager"
Sep 17 20:49:19.388518 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.388475    2587 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 20:49:19.388518 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.388494    2587 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 17 20:49:19.388654 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.388585    2587 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 17 20:49:19.393038 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393024    2587 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 20:49:19.393165 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393154    2587 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 17 20:49:19.393221 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393210    2587 kubelet.go:486] "Attempting to sync node with API server"
Sep 17 20:49:19.393279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393224    2587 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 17 20:49:19.393279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393245    2587 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 17 20:49:19.393279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393260    2587 kubelet.go:398] "Adding apiserver pod source"
Sep 17 20:49:19.393279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393273    2587 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 17 20:49:19.393706 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393688    2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 20:49:19.393770 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.393715    2587 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 17 20:49:19.395618 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.395602    2587 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 17 20:49:19.397928 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.397914    2587 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 17 20:49:19.397996 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.397956    2587 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 17 20:49:19.399192 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399179    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399198    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399207    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399215    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399226    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399232    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399244    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 17 20:49:19.399248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399249    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 17 20:49:19.399451 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399256    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 17 20:49:19.399451 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399262    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 17 20:49:19.399451 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399274    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 17 20:49:19.399451 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399279    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 17 20:49:19.399894 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399885    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 17 20:49:19.399926 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.399896    2587 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 17 20:49:19.404495 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.404474    2587 server.go:1268] "Started kubelet"
Sep 17 20:49:19.404580 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.404508    2587 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-134.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 17 20:49:19.404709 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.404637    2587 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 17 20:49:19.404812 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.404707    2587 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 17 20:49:19.405040 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.404725    2587 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 17 20:49:19.405151 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.405120    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 17 20:49:19.405151 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.405119    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 17 20:49:19.405383 ip-10-0-138-134 systemd[1]: Started Kubernetes Kubelet.
Sep 17 20:49:19.406244 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.406224    2587 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 17 20:49:19.406876 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.406862    2587 server.go:310] "Adding debug handlers to kubelet server"
Sep 17 20:49:19.409744 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.408990    2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-134.ec2.internal.18d636cdc9938749  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-134.ec2.internal,UID:ip-10-0-138-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-134.ec2.internal,},FirstTimestamp:2026-09-17 20:49:19.404435273 +0000 UTC m=+0.300823363,LastTimestamp:2026-09-17 20:49:19.404435273 +0000 UTC m=+0.300823363,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-134.ec2.internal,}"
Sep 17 20:49:19.410308 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.410296    2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kcvsp"
Sep 17 20:49:19.411039 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.411017    2587 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 17 20:49:19.413470 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.413450    2587 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 17 20:49:19.413913 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.413891    2587 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 17 20:49:19.414030 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.414011    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.414495 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414475    2587 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 17 20:49:19.414592 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414479    2587 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 17 20:49:19.414592 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414509    2587 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 17 20:49:19.414699 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414669    2587 reconstruct.go:97] "Volume reconstruction finished"
Sep 17 20:49:19.414699 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414679    2587 reconciler.go:29] "Reconciler: start to sync state"
Sep 17 20:49:19.414776 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414759    2587 factory.go:55] Registering systemd factory
Sep 17 20:49:19.414776 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414771    2587 factory.go:223] Registration of the systemd container factory successfully
Sep 17 20:49:19.414967 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414954    2587 factory.go:153] Registering CRI-O factory
Sep 17 20:49:19.415024 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.414969    2587 factory.go:223] Registration of the crio container factory successfully
Sep 17 20:49:19.415024 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.415011    2587 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 17 20:49:19.415024 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.415025    2587 factory.go:103] Registering Raw factory
Sep 17 20:49:19.415136 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.415035    2587 manager.go:1196] Started watching for new ooms in manager
Sep 17 20:49:19.415507 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.415494    2587 manager.go:349] Starting recovery of all containers
Sep 17 20:49:19.418910 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.418887    2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kcvsp"
Sep 17 20:49:19.419700 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.419496    2587 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 17 20:49:19.419700 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.419607    2587 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-134.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 17 20:49:19.429558 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.429541    2587 manager.go:354] Recovery completed
Sep 17 20:49:19.434068 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.434055    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.435889 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.435869    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.435948 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.435897    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.435948 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.435908    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.436449 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.436435    2587 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 17 20:49:19.436449 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.436444    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.436449 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.436449    2587 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 17 20:49:19.436592 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.436468    2587 state_mem.go:36] "Initialized new in-memory state store"
Sep 17 20:49:19.437086 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.437072    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.437130 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.437092    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.437130 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.437102    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.437944 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.437857    2587 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-134.ec2.internal.18d636cdcb7368f0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-134.ec2.internal,UID:ip-10-0-138-134.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-134.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-134.ec2.internal,},FirstTimestamp:2026-09-17 20:49:19.435884784 +0000 UTC m=+0.332272869,LastTimestamp:2026-09-17 20:49:19.435884784 +0000 UTC m=+0.332272869,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-134.ec2.internal,}"
Sep 17 20:49:19.438529 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.438516    2587 policy_none.go:49] "None policy: Start"
Sep 17 20:49:19.438585 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.438535    2587 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 17 20:49:19.438585 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.438547    2587 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 17 20:49:19.440507 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.440494    2587 policy_none.go:47] "Start"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486028    2587 manager.go:337] "Starting Device Plugin manager"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.486077    2587 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486090    2587 server.go:87] "Starting device plugin registration server"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486501    2587 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486515    2587 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486617    2587 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486702    2587 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.486711    2587 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.487265    2587 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 17 20:49:19.501635 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.487321    2587 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.554718 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.554672    2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 17 20:49:19.556045 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.556022    2587 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 17 20:49:19.556153 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.556085    2587 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 17 20:49:19.556153 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.556110    2587 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 17 20:49:19.556242 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.556151    2587 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 17 20:49:19.558265 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.558250    2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:19.587049 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.586986    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.587940 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.587917    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.588052 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.587954    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.588052 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.587974    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.588052 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.587998    2587 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.595718 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.595699    2587 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.595780 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.595725    2587 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-134.ec2.internal\": node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.613710 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.613688    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.656519 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.656479    2587 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"]
Sep 17 20:49:19.656656 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.656559    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.658871 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.658851    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.658939 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.658885    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.658939 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.658895    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.661117 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661106    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.661220 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661205    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.661271 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661233    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.661835 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661820    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.661907 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661842    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.661907 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661823    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.661907 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661875    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.661907 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661886    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.661907 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.661854    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.663946 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.663927    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.664033 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.663952    2587 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 17 20:49:19.664595 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.664578    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientMemory"
Sep 17 20:49:19.664676 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.664607    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasNoDiskPressure"
Sep 17 20:49:19.664676 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.664617    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeHasSufficientPID"
Sep 17 20:49:19.688996 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.688975    2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-134.ec2.internal\" not found" node="ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.693573 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.693559    2587 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-134.ec2.internal\" not found" node="ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.714032 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.714015    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.716237 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.716214    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.716321 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.716246    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.716321 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.716281    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54325897c1c76625ff4176982d832399-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"54325897c1c76625ff4176982d832399\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.814910 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.814881    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.817092 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817075    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.817136 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817102    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.817136 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817118    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54325897c1c76625ff4176982d832399-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"54325897c1c76625ff4176982d832399\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.817199 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817152    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/54325897c1c76625ff4176982d832399-config\") pod \"kube-apiserver-proxy-ip-10-0-138-134.ec2.internal\" (UID: \"54325897c1c76625ff4176982d832399\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.817199 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817158    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.817199 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.817176    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/273d394968fc451dc225e0fc61d3a80c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal\" (UID: \"273d394968fc451dc225e0fc61d3a80c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.915626 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:19.915559    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:19.992923 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.992896    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:19.997249 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:19.997224    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:20.016578 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.016554    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.118401 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.118353    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.218535 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.218458    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.319308 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.319275    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.356254 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.356225    2587 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 17 20:49:20.356696 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.356409    2587 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 17 20:49:20.419643 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.419613    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.425334 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.425305    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 20:44:19 +0000 UTC" deadline="2028-04-19 14:19:09.886658145 +0000 UTC"
Sep 17 20:49:20.425404 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.425334    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13913h29m49.461327328s"
Sep 17 20:49:20.520387 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.520312    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.555844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.555811    2587 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:20.620556 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:20.620518    2587 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-134.ec2.internal\" not found"
Sep 17 20:49:20.632956 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.632932    2587 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:20.714917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.714876    2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:20.728993 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.728966    2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 20:49:20.729792 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.729777    2587 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal"
Sep 17 20:49:20.737462 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.737442    2587 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 17 20:49:20.773922 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:20.773715    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273d394968fc451dc225e0fc61d3a80c.slice/crio-1991e5e248ca338d5a0eaabd5c9000d99e9dba995f29f3b2996daf981e12d930 WatchSource:0}: Error finding container 1991e5e248ca338d5a0eaabd5c9000d99e9dba995f29f3b2996daf981e12d930: Status 404 returned error can't find the container with id 1991e5e248ca338d5a0eaabd5c9000d99e9dba995f29f3b2996daf981e12d930
Sep 17 20:49:20.774890 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:20.774862    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54325897c1c76625ff4176982d832399.slice/crio-59f5cd9f1586bd75fc3d108e56a32b2f9c6637de281e18de757d40cb2f7667d9 WatchSource:0}: Error finding container 59f5cd9f1586bd75fc3d108e56a32b2f9c6637de281e18de757d40cb2f7667d9: Status 404 returned error can't find the container with id 59f5cd9f1586bd75fc3d108e56a32b2f9c6637de281e18de757d40cb2f7667d9
Sep 17 20:49:20.779366 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:20.779344    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 20:49:21.021005 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.020972    2587 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:21.396225 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.396193    2587 apiserver.go:52] "Watching apiserver"
Sep 17 20:49:21.406560 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.406531    2587 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 17 20:49:21.408248 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.407768    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh","openshift-cluster-node-tuning-operator/tuned-nm2dc","openshift-dns/node-resolver-9nksj","openshift-multus/multus-additional-cni-plugins-2m5mc","openshift-multus/network-metrics-daemon-ldldg","openshift-network-operator/iptables-alerter-crjfc","openshift-ovn-kubernetes/ovnkube-node-z9wf9","kube-system/global-pull-secret-syncer-fjsrk","kube-system/konnectivity-agent-8zpxn","openshift-image-registry/node-ca-bn9d2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal","openshift-multus/multus-b6fzd","openshift-network-diagnostics/network-check-target-46x9r"]
Sep 17 20:49:21.411935 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.410979    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.414428 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.414201    2587 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 17 20:49:21.415021 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.414650    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.416806 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.416510    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.417598 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417257    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8pwr6\""
Sep 17 20:49:21.417598 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417280    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ndhk6\""
Sep 17 20:49:21.417598 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417402    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 17 20:49:21.417598 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417531    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.417858 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417670    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.417858 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.417761    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.418516    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.418743    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.419160    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.419404    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.420000    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vjlxj\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.420021    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.421280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.420268    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.422929 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.422778    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.427230 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427003    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-socket-dir-parent\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.427230 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427055    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-netns\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.427230 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427093    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-run\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.427230 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427124    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-lib-modules\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.427230 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427157    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-host\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.427547 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.427321    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 17 20:49:21.428065 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428041    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428161    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mrn4\" (UniqueName: \"kubernetes.io/projected/61eaf658-5e34-4b05-9574-434d36618d28-kube-api-access-8mrn4\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428213    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-multus-certs\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428247    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-var-lib-kubelet\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428283    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-system-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428321    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-cni-binary-copy\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428345    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-conf-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428397    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428516    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-modprobe-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428567    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysconfig\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428598    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-etc-tuned\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428627    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-tmp\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428638    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428655    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37717913-b8c4-42e0-944a-7bab439f4093-host-slash\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428688    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-kubelet\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428718    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-multus\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.428720    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428748    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-etc-kubernetes\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.429702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428772    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-kubernetes\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428914    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-systemd\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428944    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-sys\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.428972    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-cnibin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429003    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429033    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-conf\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429188    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37717913-b8c4-42e0-944a-7bab439f4093-iptables-alerter-script\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429473    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmd97\" (UniqueName: \"kubernetes.io/projected/37717913-b8c4-42e0-944a-7bab439f4093-kube-api-access-mmd97\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429541    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-bin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429593    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-hostroot\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429657    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzd6\" (UniqueName: \"kubernetes.io/projected/9d00bb16-e138-4456-828c-916737f65dc4-kube-api-access-9vzd6\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429681    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wtrqz\""
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429704    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-k8s-cni-cncf-io\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429741    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-multus-daemon-config\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429797    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 17 20:49:21.430643 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.429821    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.431626 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.431605    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.431714 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.431651    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wrn9l\""
Sep 17 20:49:21.431777 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.431710    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:21.431848 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.431826    2587 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 17 20:49:21.436067 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.436046    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:21.436178 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.436110    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:21.438822 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.438800    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.442223 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.441964    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.442223 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.442069    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.442223 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.442144    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 17 20:49:21.442418 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.442258    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.442418 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.442313    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gpxh2\""
Sep 17 20:49:21.444016 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.443995    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.447019 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.446975    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.453479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.452725    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 17 20:49:21.453479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.452745    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 17 20:49:21.453479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.453008    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 17 20:49:21.453479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.453181    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.453479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.453384    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.453801 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.453624    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 17 20:49:21.453857 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.453816    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 17 20:49:21.454372 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454222    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-7ck58\""
Sep 17 20:49:21.454372 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454255    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 17 20:49:21.454372 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454279    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 17 20:49:21.455941 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454222    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 17 20:49:21.455941 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454399    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 17 20:49:21.455941 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454516    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4l9rs\""
Sep 17 20:49:21.455941 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.454539    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-m5c6c\""
Sep 17 20:49:21.473768 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.473745    2587 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bskqq"
Sep 17 20:49:21.480688 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.480665    2587 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bskqq"
Sep 17 20:49:21.515765 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.515737    2587 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 17 20:49:21.530327 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.530303    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.530513 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.530488    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37717913-b8c4-42e0-944a-7bab439f4093-iptables-alerter-script\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.530600 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.530530    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.531180 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531153    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/37717913-b8c4-42e0-944a-7bab439f4093-iptables-alerter-script\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.531180 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531156    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmd97\" (UniqueName: \"kubernetes.io/projected/37717913-b8c4-42e0-944a-7bab439f4093-kube-api-access-mmd97\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.531349 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531220    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-hostroot\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531349 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531253    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-device-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.531349 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531292    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-sys-fs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.531349 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531320    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.531349 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531329    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-hostroot\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531349    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531376    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-node-log\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531406    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-k8s-cni-cncf-io\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531464    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-dbus\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531515    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-k8s-cni-cncf-io\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531601    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdz2g\" (UniqueName: \"kubernetes.io/projected/602783c1-0d44-4f67-ab95-e2c99cd0fe21-kube-api-access-bdz2g\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531639    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531680    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-host\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531706    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-netd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531786    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd56693-7499-45af-acb5-fac485d47c8f-konnectivity-ca\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531801    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.531843 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531817    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-multus-certs\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531844    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-var-lib-kubelet\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531785    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-host\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531901    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-multus-certs\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531908    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-var-lib-kubelet\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531963    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.531995    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-etc-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532032    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsr9s\" (UniqueName: \"kubernetes.io/projected/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-kube-api-access-bsr9s\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532078    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-conf-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532140 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532110    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-etc-tuned\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532151    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-tmp\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532156    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-conf-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532187    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-log-socket\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532232    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd56693-7499-45af-acb5-fac485d47c8f-agent-certs\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532258    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37717913-b8c4-42e0-944a-7bab439f4093-host-slash\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532286    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-kubelet\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532314    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-registration-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532335    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/37717913-b8c4-42e0-944a-7bab439f4093-host-slash\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532340    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-systemd-units\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532386    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-kubelet\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532386    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-system-cni-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532431    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-multus\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532464    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-etc-kubernetes\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532495    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-multus\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532534    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/602783c1-0d44-4f67-ab95-e2c99cd0fe21-hosts-file\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532559    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:21.532644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532592    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-cnibin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532613    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-slash\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532636    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532639    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-etc-kubernetes\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532671    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-bin\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532708    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-cnibin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532721    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovn-node-metrics-cert\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532762    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-script-lib\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532797    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr9m6\" (UniqueName: \"kubernetes.io/projected/d0263dac-21f2-4cbf-a262-5aaff8eae17d-kube-api-access-hr9m6\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532827    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-conf\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532799    2587 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532860    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-bin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532894    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzd6\" (UniqueName: \"kubernetes.io/projected/9d00bb16-e138-4456-828c-916737f65dc4-kube-api-access-9vzd6\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.532899    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-var-lib-cni-bin\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533018    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysctl-conf\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533025    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtvbs\" (UniqueName: \"kubernetes.io/projected/26342db9-5ff2-4083-a42f-27bf9d6498ac-kube-api-access-mtvbs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533064    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.533432 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533089    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.534298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533129    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.534298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.533157    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-multus-daemon-config\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.534298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534260    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-socket-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.534441 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534342    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-netns\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.534441 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534373    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-var-lib-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.534441 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534406    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-multus-daemon-config\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.534616 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534437    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cnibin\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.534616 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534520    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.534616 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534558    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-kubelet-config\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.534759 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534623    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-socket-dir-parent\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.534759 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534664    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-netns\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.534759 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534706    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-run\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.534759 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534725    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-multus-socket-dir-parent\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.534759 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534753    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-lib-modules\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.534993 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534788    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mrn4\" (UniqueName: \"kubernetes.io/projected/61eaf658-5e34-4b05-9574-434d36618d28-kube-api-access-8mrn4\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.534993 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534832    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-ovn\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.534993 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534846    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-run\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.534993 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534893    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.535198 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.535063    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-lib-modules\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.535351 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.534791    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-host-run-netns\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537319    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfccg\" (UniqueName: \"kubernetes.io/projected/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-kube-api-access-vfccg\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537370    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-systemd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537405    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602783c1-0d44-4f67-ab95-e2c99cd0fe21-tmp-dir\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537442    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-system-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537468    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-cni-binary-copy\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537520    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-modprobe-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537553    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysconfig\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537584    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w44q6\" (UniqueName: \"kubernetes.io/projected/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-kube-api-access-w44q6\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537617    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-kubelet\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537715    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d00bb16-e138-4456-828c-916737f65dc4-system-cni-dir\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537760    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-kubernetes\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537797    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-systemd\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537825    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-sys\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537862    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537899    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-host\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537933    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.538917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537967    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-config\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.537996    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-env-overrides\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538029    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-serviceca\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538144    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d00bb16-e138-4456-828c-916737f65dc4-cni-binary-copy\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538212    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-kubernetes\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538260    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-modprobe-d\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538276    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-systemd\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538316    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-etc-sysconfig\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.540171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.538333    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61eaf658-5e34-4b05-9574-434d36618d28-sys\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.541117 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.541095    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-etc-tuned\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.541217 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.541166    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61eaf658-5e34-4b05-9574-434d36618d28-tmp\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.542942 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.542874    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmd97\" (UniqueName: \"kubernetes.io/projected/37717913-b8c4-42e0-944a-7bab439f4093-kube-api-access-mmd97\") pod \"iptables-alerter-crjfc\" (UID: \"37717913-b8c4-42e0-944a-7bab439f4093\") " pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.549136 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.549070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzd6\" (UniqueName: \"kubernetes.io/projected/9d00bb16-e138-4456-828c-916737f65dc4-kube-api-access-9vzd6\") pod \"multus-b6fzd\" (UID: \"9d00bb16-e138-4456-828c-916737f65dc4\") " pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.556988 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.556963    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mrn4\" (UniqueName: \"kubernetes.io/projected/61eaf658-5e34-4b05-9574-434d36618d28-kube-api-access-8mrn4\") pod \"tuned-nm2dc\" (UID: \"61eaf658-5e34-4b05-9574-434d36618d28\") " pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.560847 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.560800    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" event={"ID":"54325897c1c76625ff4176982d832399","Type":"ContainerStarted","Data":"59f5cd9f1586bd75fc3d108e56a32b2f9c6637de281e18de757d40cb2f7667d9"}
Sep 17 20:49:21.562196 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.562170    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"273d394968fc451dc225e0fc61d3a80c","Type":"ContainerStarted","Data":"1991e5e248ca338d5a0eaabd5c9000d99e9dba995f29f3b2996daf981e12d930"}
Sep 17 20:49:21.638675 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638639    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638689    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-etc-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638717    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsr9s\" (UniqueName: \"kubernetes.io/projected/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-kube-api-access-bsr9s\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638749    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-log-socket\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638774    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd56693-7499-45af-acb5-fac485d47c8f-agent-certs\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638774    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-etc-selinux\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638799    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-registration-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638828    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-systemd-units\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.638844 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638839    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-log-socket\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638853    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-system-cni-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638882    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/602783c1-0d44-4f67-ab95-e2c99cd0fe21-hosts-file\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638907    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638932    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-slash\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638955    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638979    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-bin\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639003    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovn-node-metrics-cert\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639025    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-script-lib\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639050    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr9m6\" (UniqueName: \"kubernetes.io/projected/d0263dac-21f2-4cbf-a262-5aaff8eae17d-kube-api-access-hr9m6\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639081    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtvbs\" (UniqueName: \"kubernetes.io/projected/26342db9-5ff2-4083-a42f-27bf9d6498ac-kube-api-access-mtvbs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639106    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639131    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639161    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638798    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-etc-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639190    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-socket-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639234    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-netns\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.639288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639253    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-var-lib-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639271    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cnibin\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639300    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639317    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-socket-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639336    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-kubelet-config\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639346    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-var-lib-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639367    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-slash\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639384    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-ovn\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639403    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-kubelet-config\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639408    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-openvswitch\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639416    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639269    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-system-cni-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639445    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-bin\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639464    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfccg\" (UniqueName: \"kubernetes.io/projected/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-kube-api-access-vfccg\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639480    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cnibin\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639510    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-systemd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639547    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602783c1-0d44-4f67-ab95-e2c99cd0fe21-tmp-dir\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.640215 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639581    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w44q6\" (UniqueName: \"kubernetes.io/projected/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-kube-api-access-w44q6\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639606    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-kubelet\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639633    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639656    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-host\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639681    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639707    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-config\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639731    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-env-overrides\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639772    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-serviceca\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639805    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-device-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639832    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-sys-fs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639859    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639890    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639917    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-node-log\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639948    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-dbus\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639976    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdz2g\" (UniqueName: \"kubernetes.io/projected/602783c1-0d44-4f67-ab95-e2c99cd0fe21-kube-api-access-bdz2g\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640007    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-netd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.638885    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-registration-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.641091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640037    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd56693-7499-45af-acb5-fac485d47c8f-konnectivity-ca\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.639305    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-systemd-units\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640095    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640105    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-netns\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640114    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/602783c1-0d44-4f67-ab95-e2c99cd0fe21-hosts-file\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640144    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-run-ovn-kubernetes\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640010    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-binary-copy\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640376    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-ovn\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640647    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.640742    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.640810    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:22.140788031 +0000 UTC m=+3.037176130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640909    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-serviceca\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.640966    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-sys-fs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641002    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-run-systemd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641017    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-node-log\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641031    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-script-lib\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641157    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/658b0509-365d-4197-bb9e-682ffdccacb2-dbus\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:21.641791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641238    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/602783c1-0d44-4f67-ab95-e2c99cd0fe21-tmp-dir\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641368    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-kubelet\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641423    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d0263dac-21f2-4cbf-a262-5aaff8eae17d-host-cni-netd\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.641436    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.641483    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:22.141467505 +0000 UTC m=+3.037855588 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641535    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-host\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641581    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641697    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/26342db9-5ff2-4083-a42f-27bf9d6498ac-device-dir\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641863    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovn-node-metrics-cert\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.641929    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.642070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.642073    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bfd56693-7499-45af-acb5-fac485d47c8f-agent-certs\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.642093    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-ovnkube-config\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.642133    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d0263dac-21f2-4cbf-a262-5aaff8eae17d-env-overrides\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.642740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.642142    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bfd56693-7499-45af-acb5-fac485d47c8f-konnectivity-ca\") pod \"konnectivity-agent-8zpxn\" (UID: \"bfd56693-7499-45af-acb5-fac485d47c8f\") " pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.655305 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.654890    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:21.655305 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.654917    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:21.655305 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.654932    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:21.655305 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:21.655015    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:22.154987207 +0000 UTC m=+3.051375286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:21.655305 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.655194    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdz2g\" (UniqueName: \"kubernetes.io/projected/602783c1-0d44-4f67-ab95-e2c99cd0fe21-kube-api-access-bdz2g\") pod \"node-resolver-9nksj\" (UID: \"602783c1-0d44-4f67-ab95-e2c99cd0fe21\") " pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.655689 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.655317    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w44q6\" (UniqueName: \"kubernetes.io/projected/3c6387a1-2f78-4e35-a245-65dfe1c0dea5-kube-api-access-w44q6\") pod \"node-ca-bn9d2\" (UID: \"3c6387a1-2f78-4e35-a245-65dfe1c0dea5\") " pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.655689 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.655448    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfccg\" (UniqueName: \"kubernetes.io/projected/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-kube-api-access-vfccg\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:21.657905 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.657878    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsr9s\" (UniqueName: \"kubernetes.io/projected/9a43e2b1-ae80-4ced-9db5-9a82f26503cb-kube-api-access-bsr9s\") pod \"multus-additional-cni-plugins-2m5mc\" (UID: \"9a43e2b1-ae80-4ced-9db5-9a82f26503cb\") " pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.659150 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.659123    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtvbs\" (UniqueName: \"kubernetes.io/projected/26342db9-5ff2-4083-a42f-27bf9d6498ac-kube-api-access-mtvbs\") pod \"aws-ebs-csi-driver-node-bg7xh\" (UID: \"26342db9-5ff2-4083-a42f-27bf9d6498ac\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.660583 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.660559    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr9m6\" (UniqueName: \"kubernetes.io/projected/d0263dac-21f2-4cbf-a262-5aaff8eae17d-kube-api-access-hr9m6\") pod \"ovnkube-node-z9wf9\" (UID: \"d0263dac-21f2-4cbf-a262-5aaff8eae17d\") " pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:21.725463 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.725411    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-crjfc"
Sep 17 20:49:21.737364 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.737272    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-b6fzd"
Sep 17 20:49:21.738191 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:21.738161    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37717913_b8c4_42e0_944a_7bab439f4093.slice/crio-4b0795b296b17e2402ad9ec91d3372bc141a52ac4b8b0b694722d697d2988d69 WatchSource:0}: Error finding container 4b0795b296b17e2402ad9ec91d3372bc141a52ac4b8b0b694722d697d2988d69: Status 404 returned error can't find the container with id 4b0795b296b17e2402ad9ec91d3372bc141a52ac4b8b0b694722d697d2988d69
Sep 17 20:49:21.746847 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.746646    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nm2dc"
Sep 17 20:49:21.748471 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:21.748450    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d00bb16_e138_4456_828c_916737f65dc4.slice/crio-b94700a408ea45fdc0f3aa927e153f6f069f6a375e69541603fe5b99f70994dd WatchSource:0}: Error finding container b94700a408ea45fdc0f3aa927e153f6f069f6a375e69541603fe5b99f70994dd: Status 404 returned error can't find the container with id b94700a408ea45fdc0f3aa927e153f6f069f6a375e69541603fe5b99f70994dd
Sep 17 20:49:21.753209 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.753188    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9nksj"
Sep 17 20:49:21.762336 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.762307    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2m5mc"
Sep 17 20:49:21.766730 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.766704    2587 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 17 20:49:21.771263 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.771241    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh"
Sep 17 20:49:21.780282 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.780260    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:21.788092 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.788063    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bn9d2"
Sep 17 20:49:21.794146 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:21.794122    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:22.144331 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.144241    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:22.144515 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.144352    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:22.144515 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.144416    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:22.144515 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.144489    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:22.144515 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.144515    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:23.144492095 +0000 UTC m=+4.040880173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:22.144742 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.144549    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:23.144531762 +0000 UTC m=+4.040919854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:22.245111 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.245072    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:22.245293 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.245248    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:22.245293 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.245269    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:22.245293 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.245281    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:22.245479 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.245346    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:23.245327384 +0000 UTC m=+4.141715464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:22.484044 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.483964    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 20:44:21 +0000 UTC" deadline="2028-07-01 05:50:02.194007516 +0000 UTC"
Sep 17 20:49:22.484044 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.484001    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h0m39.710009938s"
Sep 17 20:49:22.557251 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.557217    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:22.557458 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:22.557351    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:22.565516 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.565462    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b6fzd" event={"ID":"9d00bb16-e138-4456-828c-916737f65dc4","Type":"ContainerStarted","Data":"b94700a408ea45fdc0f3aa927e153f6f069f6a375e69541603fe5b99f70994dd"}
Sep 17 20:49:22.568531 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:22.567769    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crjfc" event={"ID":"37717913-b8c4-42e0-944a-7bab439f4093","Type":"ContainerStarted","Data":"4b0795b296b17e2402ad9ec91d3372bc141a52ac4b8b0b694722d697d2988d69"}
Sep 17 20:49:22.940296 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.940263    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61eaf658_5e34_4b05_9574_434d36618d28.slice/crio-ca594b15d83732c20e8e4a3d3e0667843909f47fad5498e247947865abac048d WatchSource:0}: Error finding container ca594b15d83732c20e8e4a3d3e0667843909f47fad5498e247947865abac048d: Status 404 returned error can't find the container with id ca594b15d83732c20e8e4a3d3e0667843909f47fad5498e247947865abac048d
Sep 17 20:49:22.956958 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.956928    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfd56693_7499_45af_acb5_fac485d47c8f.slice/crio-24dc9515641831603bf4c571b7e4f06026a5da22ef424c593971a9e4f514ee7f WatchSource:0}: Error finding container 24dc9515641831603bf4c571b7e4f06026a5da22ef424c593971a9e4f514ee7f: Status 404 returned error can't find the container with id 24dc9515641831603bf4c571b7e4f06026a5da22ef424c593971a9e4f514ee7f
Sep 17 20:49:22.958411 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.958382    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a43e2b1_ae80_4ced_9db5_9a82f26503cb.slice/crio-d16b543aefbf2d894b548c18e3018181ae55db6ef293da539b384bf7cfd00a3f WatchSource:0}: Error finding container d16b543aefbf2d894b548c18e3018181ae55db6ef293da539b384bf7cfd00a3f: Status 404 returned error can't find the container with id d16b543aefbf2d894b548c18e3018181ae55db6ef293da539b384bf7cfd00a3f
Sep 17 20:49:22.959458 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.959400    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26342db9_5ff2_4083_a42f_27bf9d6498ac.slice/crio-e36818466b9d4d7dd2755b142939a0d23c4944e7f382245cddd60c59e36fbfc3 WatchSource:0}: Error finding container e36818466b9d4d7dd2755b142939a0d23c4944e7f382245cddd60c59e36fbfc3: Status 404 returned error can't find the container with id e36818466b9d4d7dd2755b142939a0d23c4944e7f382245cddd60c59e36fbfc3
Sep 17 20:49:22.961852 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.961827    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c6387a1_2f78_4e35_a245_65dfe1c0dea5.slice/crio-ec48bb2f031bb5d5ae91c27be2c251b4180217f61cc54abae7a168820d36a08e WatchSource:0}: Error finding container ec48bb2f031bb5d5ae91c27be2c251b4180217f61cc54abae7a168820d36a08e: Status 404 returned error can't find the container with id ec48bb2f031bb5d5ae91c27be2c251b4180217f61cc54abae7a168820d36a08e
Sep 17 20:49:22.964464 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.963146    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0263dac_21f2_4cbf_a262_5aaff8eae17d.slice/crio-534e54ef5bf56fbce4f00fb65ec0ad0d988d5d54b332a5143ba5b20afddf791a WatchSource:0}: Error finding container 534e54ef5bf56fbce4f00fb65ec0ad0d988d5d54b332a5143ba5b20afddf791a: Status 404 returned error can't find the container with id 534e54ef5bf56fbce4f00fb65ec0ad0d988d5d54b332a5143ba5b20afddf791a
Sep 17 20:49:22.965186 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:22.964874    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod602783c1_0d44_4f67_ab95_e2c99cd0fe21.slice/crio-dd9486c4fba8002a68e86aaef47ca7171b61cef3afce604a3960074ab2dc3281 WatchSource:0}: Error finding container dd9486c4fba8002a68e86aaef47ca7171b61cef3afce604a3960074ab2dc3281: Status 404 returned error can't find the container with id dd9486c4fba8002a68e86aaef47ca7171b61cef3afce604a3960074ab2dc3281
Sep 17 20:49:23.153028 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.152994    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:23.153201 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.153046    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:23.153201 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.153178    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:23.153201 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.153186    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:23.153374 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.153244    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:25.153227613 +0000 UTC m=+6.049615708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:23.153374 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.153264    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:25.153254244 +0000 UTC m=+6.049642326 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:23.253762 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.253728    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:23.253995 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.253973    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:23.254102 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.253997    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:23.254102 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.254011    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:23.254102 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.254077    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:25.254054475 +0000 UTC m=+6.150442566 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:23.484979 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.484864    2587 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 20:44:21 +0000 UTC" deadline="2028-06-13 04:53:22.76653987 +0000 UTC"
Sep 17 20:49:23.484979 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.484931    2587 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15224h3m59.281612566s"
Sep 17 20:49:23.557747 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.557000    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:23.557747 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.557133    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:23.557747 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.557740    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:23.558110 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:23.557853    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:23.578059 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.578026    2587 generic.go:356] "Generic (PLEG): container finished" podID="273d394968fc451dc225e0fc61d3a80c" containerID="83f63e212fc484f475734ea1ac0afda52cec9325638ff28c761602bb9c8f5bd6" exitCode=0
Sep 17 20:49:23.578736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.578709    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"273d394968fc451dc225e0fc61d3a80c","Type":"ContainerDied","Data":"83f63e212fc484f475734ea1ac0afda52cec9325638ff28c761602bb9c8f5bd6"}
Sep 17 20:49:23.582336 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.581932    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9nksj" event={"ID":"602783c1-0d44-4f67-ab95-e2c99cd0fe21","Type":"ContainerStarted","Data":"dd9486c4fba8002a68e86aaef47ca7171b61cef3afce604a3960074ab2dc3281"}
Sep 17 20:49:23.586499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.586324    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bn9d2" event={"ID":"3c6387a1-2f78-4e35-a245-65dfe1c0dea5","Type":"ContainerStarted","Data":"ec48bb2f031bb5d5ae91c27be2c251b4180217f61cc54abae7a168820d36a08e"}
Sep 17 20:49:23.591724 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.591701    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh" event={"ID":"26342db9-5ff2-4083-a42f-27bf9d6498ac","Type":"ContainerStarted","Data":"e36818466b9d4d7dd2755b142939a0d23c4944e7f382245cddd60c59e36fbfc3"}
Sep 17 20:49:23.600104 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.600074    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zpxn" event={"ID":"bfd56693-7499-45af-acb5-fac485d47c8f","Type":"ContainerStarted","Data":"24dc9515641831603bf4c571b7e4f06026a5da22ef424c593971a9e4f514ee7f"}
Sep 17 20:49:23.612178 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.612143    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nm2dc" event={"ID":"61eaf658-5e34-4b05-9574-434d36618d28","Type":"ContainerStarted","Data":"ca594b15d83732c20e8e4a3d3e0667843909f47fad5498e247947865abac048d"}
Sep 17 20:49:23.618171 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.618137    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" event={"ID":"54325897c1c76625ff4176982d832399","Type":"ContainerStarted","Data":"0b488759088d636e3a4e5a7118b27a434ef42787639444001fc3e382a9d93964"}
Sep 17 20:49:23.625166 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.625138    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"534e54ef5bf56fbce4f00fb65ec0ad0d988d5d54b332a5143ba5b20afddf791a"}
Sep 17 20:49:23.634641 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:23.634610    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerStarted","Data":"d16b543aefbf2d894b548c18e3018181ae55db6ef293da539b384bf7cfd00a3f"}
Sep 17 20:49:24.556778 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:24.556506    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:24.557237 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:24.556870    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:24.664006 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:24.661561    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" event={"ID":"273d394968fc451dc225e0fc61d3a80c","Type":"ContainerStarted","Data":"6872a3b9fbc0f5efb764d3d09041c7e35256e8a7f09a03df581e2aad71ee4008"}
Sep 17 20:49:24.680071 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:24.679999    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-134.ec2.internal" podStartSLOduration=3.679955488 podStartE2EDuration="3.679955488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:24.679024804 +0000 UTC m=+5.575412907" watchObservedRunningTime="2026-09-17 20:49:24.679955488 +0000 UTC m=+5.576343613"
Sep 17 20:49:24.680313 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:24.680279    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-134.ec2.internal" podStartSLOduration=3.680267404 podStartE2EDuration="3.680267404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:23.636724912 +0000 UTC m=+4.533113012" watchObservedRunningTime="2026-09-17 20:49:24.680267404 +0000 UTC m=+5.576655503"
Sep 17 20:49:25.169815 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:25.169776    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:25.169999 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:25.169838    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:25.169999 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.169977    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:25.170141 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.170043    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.170023949 +0000 UTC m=+10.066412038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:25.170543 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.170464    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:25.170543 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.170539    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.170524501 +0000 UTC m=+10.066912590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:25.270940 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:25.270899    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:25.271121 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.271101    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:25.271206 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.271131    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:25.271206 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.271145    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:25.271270 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.271212    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:29.271191787 +0000 UTC m=+10.167579866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:25.558800 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:25.556963    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:25.558800 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.557099    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:25.558800 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:25.557186    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:25.558800 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:25.557307    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:26.557569 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:26.557045    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:26.557569 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:26.557193    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:27.558891 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:27.556829    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:27.558891 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:27.556965    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:27.558891 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:27.557372    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:27.558891 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:27.557495    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:28.557278 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:28.557162    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:28.557439 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:28.557299    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:29.206583    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:29.206642    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.206805    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.206870    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:37.206851041 +0000 UTC m=+18.103239139 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.207028    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:29.207197 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.207102    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:37.207084376 +0000 UTC m=+18.103472454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:29.307852 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:29.307750    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:29.308010 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.307907    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:29.308010 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.307932    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:29.308010 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.307945    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:29.308010 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.308001    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:37.307984236 +0000 UTC m=+18.204372323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:29.563593 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:29.560163    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:29.563593 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.560358    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:29.563593 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:29.560859    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:29.563593 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:29.561031    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:30.556703 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:30.556620    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:30.557198 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:30.556738    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:31.556772 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:31.556736    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:31.557224 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:31.556876    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:31.557224 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:31.556932    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:31.557224 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:31.557026    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:32.557059 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:32.557012    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:32.557478 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:32.557170    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:33.557298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:33.557218    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:33.557298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:33.557250    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:33.557801 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:33.557338    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:33.557801 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:33.557480    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:34.556551 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:34.556522    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:34.556702 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:34.556631    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:35.557389 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:35.557356    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:35.557814 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:35.557499    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:35.557814 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:35.557553    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:35.557814 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:35.557712    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:36.556839 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:36.556804    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:36.557093 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:36.556940    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:37.269644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:37.269609    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:37.269644 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:37.269653    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:37.270082 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.269771    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:37.270082 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.269831    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:49:53.269813013 +0000 UTC m=+34.166201098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:37.270082 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.269771    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:37.270082 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.269903    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:49:53.269891512 +0000 UTC m=+34.166279600 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:37.370803 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:37.370734    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:37.370936 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.370868    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:37.370936 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.370882    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:37.370936 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.370890    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:37.370936 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.370938    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:49:53.370924695 +0000 UTC m=+34.267312784 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:37.556358 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:37.556319    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:37.556564 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:37.556372    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:37.556564 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.556463    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:37.556657 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:37.556581    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:38.556372 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:38.556337    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:38.556773 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:38.556459    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:39.557797 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:39.557762    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:39.558347 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:39.557831    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:39.558509 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:39.558377    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:39.558509 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:39.558434    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:40.557251 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:40.557221    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:40.557398 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:40.557315    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:41.557131 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.556744    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:41.557760 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.556747    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:41.557760 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:41.557229    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:41.557760 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:41.557324    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:41.698833 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.698784    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9nksj" event={"ID":"602783c1-0d44-4f67-ab95-e2c99cd0fe21","Type":"ContainerStarted","Data":"3c76dd56c2c362ea529cf867b69a917462e6195879fb9e042b23f005b1a8382f"}
Sep 17 20:49:41.700406 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.700378    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bn9d2" event={"ID":"3c6387a1-2f78-4e35-a245-65dfe1c0dea5","Type":"ContainerStarted","Data":"0821d48fce68bdd927021db8807610fcaf93f75b09b4ca33d8003a60784bedcb"}
Sep 17 20:49:41.702027 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.702000    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh" event={"ID":"26342db9-5ff2-4083-a42f-27bf9d6498ac","Type":"ContainerStarted","Data":"3230a3d82ec405e14f4827a840950fa39152795b8002d38936d0d4af96845079"}
Sep 17 20:49:41.703582 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.703551    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zpxn" event={"ID":"bfd56693-7499-45af-acb5-fac485d47c8f","Type":"ContainerStarted","Data":"14b257751c6f719f9d8f86aaa7c4a97798b977ba443dec22e7f4b2810c05f0f2"}
Sep 17 20:49:41.705213 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.705195    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nm2dc" event={"ID":"61eaf658-5e34-4b05-9574-434d36618d28","Type":"ContainerStarted","Data":"5406f037fae3780cca77434433aab8d6aa128312e13da3427938958524c7c977"}
Sep 17 20:49:41.706827 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.706791    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-b6fzd" event={"ID":"9d00bb16-e138-4456-828c-916737f65dc4","Type":"ContainerStarted","Data":"a9dbb1f6ed8db8dc4ba961b683989b47b5b7e2d7f82eb102b00ce881ad5f98e4"}
Sep 17 20:49:41.710207 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710188    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"5a1d0243e667f6bc3d40df76bc9c03cc2f042e59c9e38b6f3cfe10c47f192b39"}
Sep 17 20:49:41.710305 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710253    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"0f70d44692cc02235bf133cb89c3edb7dc4f9c98a6ddc063da1017673009bcca"}
Sep 17 20:49:41.710305 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710294    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"8ecbaf99a55537099269614e5d4120aa0d42b9bb97fbee1c65eb1a73213d4c40"}
Sep 17 20:49:41.710402 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710332    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"79b4d72d750845282d220039ce72cd7ffea87a605768ecc54235914cf0502389"}
Sep 17 20:49:41.710402 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710367    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"a21a7d36ab29a844ca6ebd5686f7a5e08e4586083bbc852d6086106d07ae1b7f"}
Sep 17 20:49:41.710527 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.710407    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"6851ca5f01b6ec0814b52562fc98d9e61648ad636807d615bd3f17567c2d98bb"}
Sep 17 20:49:41.711892 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.711870    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="1f27126e1f8bd26a888fac701c874987ab71ce14ea6ddd7886c756514928bd55" exitCode=0
Sep 17 20:49:41.711966 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.711904    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"1f27126e1f8bd26a888fac701c874987ab71ce14ea6ddd7886c756514928bd55"}
Sep 17 20:49:41.716405 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.716363    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9nksj" podStartSLOduration=7.656428069 podStartE2EDuration="21.71634593s" totalImagesPullingTime="14.059917861s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:41.715842637 +0000 UTC m=+22.612230763" watchObservedRunningTime="2026-09-17 20:49:41.71634593 +0000 UTC m=+22.612734026"
Sep 17 20:49:41.742355 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.742264    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8zpxn" podStartSLOduration=4.069279016 podStartE2EDuration="21.742243592s" totalImagesPullingTime="17.672964576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:41.741505318 +0000 UTC m=+22.637893413" watchObservedRunningTime="2026-09-17 20:49:41.742243592 +0000 UTC m=+22.638631688"
Sep 17 20:49:41.759369 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.759334    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nm2dc" podStartSLOduration=4.071047639 podStartE2EDuration="21.759321093s" totalImagesPullingTime="17.688273454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:41.75905584 +0000 UTC m=+22.655443933" watchObservedRunningTime="2026-09-17 20:49:41.759321093 +0000 UTC m=+22.655709188"
Sep 17 20:49:41.775488 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.775457    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bn9d2" podStartSLOduration=4.109062246 podStartE2EDuration="21.775445757s" totalImagesPullingTime="17.666383511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:41.775228331 +0000 UTC m=+22.671616427" watchObservedRunningTime="2026-09-17 20:49:41.775445757 +0000 UTC m=+22.671833846"
Sep 17 20:49:41.798687 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.798645    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-b6fzd" podStartSLOduration=2.900324913 podStartE2EDuration="21.798632834s" totalImagesPullingTime="18.898307921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:41.798240723 +0000 UTC m=+22.694628818" watchObservedRunningTime="2026-09-17 20:49:41.798632834 +0000 UTC m=+22.695020927"
Sep 17 20:49:41.882991 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:41.882965    2587 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 17 20:49:42.500442 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.500326    2587 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T20:49:41.882987416Z","UUID":"bac0b65c-eff7-49d8-9476-920afa5a34c8","Handler":null,"Name":"","Endpoint":""}
Sep 17 20:49:42.501866 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.501843    2587 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 17 20:49:42.501979 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.501881    2587 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 17 20:49:42.556945 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.556919    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:42.557123 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:42.557035    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:42.716727 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.716685    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-crjfc" event={"ID":"37717913-b8c4-42e0-944a-7bab439f4093","Type":"ContainerStarted","Data":"9288768e158b50c59d625597d039a0693e7a0254fef17c171c532e8ae43bad78"}
Sep 17 20:49:42.719612 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.719572    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh" event={"ID":"26342db9-5ff2-4083-a42f-27bf9d6498ac","Type":"ContainerStarted","Data":"8929c6bc14a0fa2e40cc43299d574b4293d593e0ae32f4c6e89f8cf338745aad"}
Sep 17 20:49:42.742262 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:42.742198    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-crjfc" podStartSLOduration=7.459659703 podStartE2EDuration="22.742178144s" totalImagesPullingTime="15.282518441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:42.741403954 +0000 UTC m=+23.637792049" watchObservedRunningTime="2026-09-17 20:49:42.742178144 +0000 UTC m=+23.638566240"
Sep 17 20:49:43.556884 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:43.556845    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:43.557061 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:43.556980    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:43.557061 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:43.557041    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:43.557188 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:43.557166    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:43.724863 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:43.724820    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh" event={"ID":"26342db9-5ff2-4083-a42f-27bf9d6498ac","Type":"ContainerStarted","Data":"f81827a0473469866b138d1178b81de2691f4dbd1ae7f877ab6f758520768bd2"}
Sep 17 20:49:43.729155 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:43.729124    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"00a26e2d485f9de21a7ac298450894b056e97d353548a8680a7d18c48912a1e2"}
Sep 17 20:49:43.748542 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:43.748487    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bg7xh" podStartSLOduration=3.958397262 podStartE2EDuration="23.748468453s" totalImagesPullingTime="19.790071191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:43.746749239 +0000 UTC m=+24.643137334" watchObservedRunningTime="2026-09-17 20:49:43.748468453 +0000 UTC m=+24.644856542"
Sep 17 20:49:44.556376 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:44.556334    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:44.556547 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:44.556450    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:45.557044 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:45.557003    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:45.557526 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:45.557135    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:45.557526 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:45.557191    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:45.557526 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:45.557318    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:46.266348 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.266119    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:46.267008 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.266989    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:46.557033 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.556948    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:46.557171 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:46.557044    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:46.739612 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.739582    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" event={"ID":"d0263dac-21f2-4cbf-a262-5aaff8eae17d","Type":"ContainerStarted","Data":"c517bd025977c087edf9d2d37ef1d57d8f6ad67909a4e6f52f877115228ca158"}
Sep 17 20:49:46.739914 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.739883    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:46.741513 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.741481    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="4cc2509a1790e2d65eb684a7dca6980b5b945c0043b2c262781e0f53b525497e" exitCode=0
Sep 17 20:49:46.741626 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.741557    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"4cc2509a1790e2d65eb684a7dca6980b5b945c0043b2c262781e0f53b525497e"}
Sep 17 20:49:46.755142 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.755120    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:46.776030 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:46.775982    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9" podStartSLOduration=8.826979163 podStartE2EDuration="26.775967696s" totalImagesPullingTime="17.948988533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:46.774583615 +0000 UTC m=+27.670971709" watchObservedRunningTime="2026-09-17 20:49:46.775967696 +0000 UTC m=+27.672355789"
Sep 17 20:49:47.556442 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:47.556378    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:47.556442 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:47.556406    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:47.556677 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:47.556551    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:47.556721 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:47.556699    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:47.744581 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:47.744553    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:47.744872 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:47.744649    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:47.763938 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:47.763742    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:49:48.036302 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.036271    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fjsrk"]
Sep 17 20:49:48.036501 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.036363    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:48.036501 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:48.036459    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:48.039053 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.039030    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-46x9r"]
Sep 17 20:49:48.039173 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.039123    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:48.039220 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:48.039194    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:48.039587 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.039561    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldldg"]
Sep 17 20:49:48.039744 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.039636    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:48.039744 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:48.039724    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:48.748228 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.748191    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="cb835db87b57a0079464a709ced95c901065c4233f9b918a766688e641d19f7d" exitCode=0
Sep 17 20:49:48.748700 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:48.748279    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"cb835db87b57a0079464a709ced95c901065c4233f9b918a766688e641d19f7d"}
Sep 17 20:49:49.557923 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.557844    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:49.558087 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:49.557929    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:49.558087 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.558041    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:49.558201 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.558163    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:49.558201 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:49.558160    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:49.558269 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:49.558227    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:49.742346 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.742311    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:49.742552 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.742461    2587 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 17 20:49:49.742945 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.742923    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8zpxn"
Sep 17 20:49:49.753534 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.753502    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="e86f137312d61d6def33990217f00baa710268057605e3e8038413aa6f356ebc" exitCode=0
Sep 17 20:49:49.753875 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:49.753593    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"e86f137312d61d6def33990217f00baa710268057605e3e8038413aa6f356ebc"}
Sep 17 20:49:51.556795 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:51.556760    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:51.557255 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:51.556813    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:51.557255 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:51.556766    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:51.557255 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:51.556893    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:51.557255 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:51.556979    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:51.557255 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:51.557082    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:52.455469 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:52.455213    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:52.560007 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:52.559969    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:52.777848 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:52.777811    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:52.875132 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:52.875098    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:53.288549 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.288512    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:53.288742 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.288567    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:53.288742 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.288662    2587 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:53.288742 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.288674    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:53.288742 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.288729    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:50:25.288711019 +0000 UTC m=+66.185099105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 17 20:49:53.288964 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.288750    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret podName:658b0509-365d-4197-bb9e-682ffdccacb2 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:25.288740049 +0000 UTC m=+66.185128135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret") pod "global-pull-secret-syncer-fjsrk" (UID: "658b0509-365d-4197-bb9e-682ffdccacb2") : object "kube-system"/"original-pull-secret" not registered
Sep 17 20:49:53.351261 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.351229    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:53.389454 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.389413    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:53.389639 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.389592    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 17 20:49:53.389639 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.389617    2587 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 17 20:49:53.389639 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.389630    2587 projected.go:196] Error preparing data for projected volume kube-api-access-5czz4 for pod openshift-network-diagnostics/network-check-target-46x9r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:53.389782 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.389695    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4 podName:d356f916-1ba5-463a-97ae-bc8030fc8d8f nodeName:}" failed. No retries permitted until 2026-09-17 20:50:25.389676466 +0000 UTC m=+66.286064541 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5czz4" (UniqueName: "kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4") pod "network-check-target-46x9r" (UID: "d356f916-1ba5-463a-97ae-bc8030fc8d8f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 17 20:49:53.556618 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.556537    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:53.556787 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.556537    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:53.556787 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.556680    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldldg" podUID="ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa"
Sep 17 20:49:53.556787 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:53.556537    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:53.556787 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.556770    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-46x9r" podUID="d356f916-1ba5-463a-97ae-bc8030fc8d8f"
Sep 17 20:49:53.556955 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:53.556801    2587 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fjsrk" podUID="658b0509-365d-4197-bb9e-682ffdccacb2"
Sep 17 20:49:54.443830 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.443808    2587 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-134.ec2.internal" event="NodeReady"
Sep 17 20:49:54.444178 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.443972    2587 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 17 20:49:54.521916 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.521887    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-brdvm"]
Sep 17 20:49:54.534802 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.534785    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2dpzv"]
Sep 17 20:49:54.534933 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.534917    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.543002 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.542980    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h6qzc\""
Sep 17 20:49:54.543098 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.542998    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 17 20:49:54.545251 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.545232    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 17 20:49:54.548906 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.548885    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brdvm"]
Sep 17 20:49:54.549014 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.548965    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.553553 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.553530    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 17 20:49:54.553674 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.553655    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gxtkj\""
Sep 17 20:49:54.553743 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.553670    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 17 20:49:54.553743 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.553666    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 17 20:49:54.554437 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.554406    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2dpzv"]
Sep 17 20:49:54.698661 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698623    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c352448-d5b6-4c8f-aeba-ead56fa5b354-cert\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.698661 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698657    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/159ae6fc-327d-498f-b31e-94a7396a9abb-tmp-dir\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.698855 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698678    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.698855 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698705    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5znw7\" (UniqueName: \"kubernetes.io/projected/2c352448-d5b6-4c8f-aeba-ead56fa5b354-kube-api-access-5znw7\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.698855 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698740    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt2xx\" (UniqueName: \"kubernetes.io/projected/159ae6fc-327d-498f-b31e-94a7396a9abb-kube-api-access-gt2xx\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.698855 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.698793    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/159ae6fc-327d-498f-b31e-94a7396a9abb-config-volume\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.768103 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.768067    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="13372e68acada94926f4d00c339a337e0d9dcadaedd621fc72909eacb00fc811" exitCode=0
Sep 17 20:49:54.768219 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.768110    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"13372e68acada94926f4d00c339a337e0d9dcadaedd621fc72909eacb00fc811"}
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800106    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c352448-d5b6-4c8f-aeba-ead56fa5b354-cert\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800140    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/159ae6fc-327d-498f-b31e-94a7396a9abb-tmp-dir\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800167    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800191    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5znw7\" (UniqueName: \"kubernetes.io/projected/2c352448-d5b6-4c8f-aeba-ead56fa5b354-kube-api-access-5znw7\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800211    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt2xx\" (UniqueName: \"kubernetes.io/projected/159ae6fc-327d-498f-b31e-94a7396a9abb-kube-api-access-gt2xx\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.800445 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800245    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/159ae6fc-327d-498f-b31e-94a7396a9abb-config-volume\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.801707 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.800799    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/159ae6fc-327d-498f-b31e-94a7396a9abb-config-volume\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.802717 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:54.802006    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:54.802717 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:54.802070    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:49:55.302051812 +0000 UTC m=+36.198439901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:54.802717 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.802484    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/159ae6fc-327d-498f-b31e-94a7396a9abb-tmp-dir\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.805925 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.805907    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2c352448-d5b6-4c8f-aeba-ead56fa5b354-cert\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.818684 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.818657    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt2xx\" (UniqueName: \"kubernetes.io/projected/159ae6fc-327d-498f-b31e-94a7396a9abb-kube-api-access-gt2xx\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:54.820280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.820256    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5znw7\" (UniqueName: \"kubernetes.io/projected/2c352448-d5b6-4c8f-aeba-ead56fa5b354-kube-api-access-5znw7\") pod \"ingress-canary-2dpzv\" (UID: \"2c352448-d5b6-4c8f-aeba-ead56fa5b354\") " pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:54.859800 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:54.859780    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2dpzv"
Sep 17 20:49:55.189762 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.189529    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2dpzv"]
Sep 17 20:49:55.192743 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:49:55.192721    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c352448_d5b6_4c8f_aeba_ead56fa5b354.slice/crio-4db40ab1eb80323eb27492bf5164c6f37a43f036166fc896b37e5dbd35dfc060 WatchSource:0}: Error finding container 4db40ab1eb80323eb27492bf5164c6f37a43f036166fc896b37e5dbd35dfc060: Status 404 returned error can't find the container with id 4db40ab1eb80323eb27492bf5164c6f37a43f036166fc896b37e5dbd35dfc060
Sep 17 20:49:55.303888 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.303850    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:55.304034 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:55.303985    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:55.304077 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:55.304056    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:49:56.304038097 +0000 UTC m=+37.200426169 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:55.556435 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.556389    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:49:55.557162 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.556451    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:49:55.557162 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.556464    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:49:55.559398 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.559378    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 20:49:55.560580 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.560558    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j729c\""
Sep 17 20:49:55.560733 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.560582    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 20:49:55.560733 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.560589    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 20:49:55.560733 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.560626    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-775ms\""
Sep 17 20:49:55.560837 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.560748    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 20:49:55.771823 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.771781    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2dpzv" event={"ID":"2c352448-d5b6-4c8f-aeba-ead56fa5b354","Type":"ContainerStarted","Data":"4db40ab1eb80323eb27492bf5164c6f37a43f036166fc896b37e5dbd35dfc060"}
Sep 17 20:49:55.775469 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.775412    2587 generic.go:356] "Generic (PLEG): container finished" podID="9a43e2b1-ae80-4ced-9db5-9a82f26503cb" containerID="239041a574e2e0fee2753047ceee76e59e0b1594139361b074cac6a935d79801" exitCode=0
Sep 17 20:49:55.775603 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.775467    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerDied","Data":"239041a574e2e0fee2753047ceee76e59e0b1594139361b074cac6a935d79801"}
Sep 17 20:49:55.811810 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.811785    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:55.882719 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:55.882681    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:56.310878 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:56.310849    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:56.311040 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:56.310975    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:56.311040 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:56.311032    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:49:58.311016202 +0000 UTC m=+39.207404275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:56.781247 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:56.781215    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" event={"ID":"9a43e2b1-ae80-4ced-9db5-9a82f26503cb","Type":"ContainerStarted","Data":"7f7258c7a7fd55747428b2e1c119b903c153c5598b6e73c607813c003a1595c2"}
Sep 17 20:49:56.937436 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:56.937385    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:57.010688 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:57.010649    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:57.785627 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:57.785589    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2dpzv" event={"ID":"2c352448-d5b6-4c8f-aeba-ead56fa5b354","Type":"ContainerStarted","Data":"c6ca4ffb0933585d3c56409e942ee9e9bdc8a3669a46f757a26f87e82e5adf30"}
Sep 17 20:49:57.813542 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:57.813486    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2m5mc" podStartSLOduration=10.137557251 podStartE2EDuration="37.813469543s" totalImagesPullingTime="27.675912292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:56.81100546 +0000 UTC m=+37.707393556" watchObservedRunningTime="2026-09-17 20:49:57.813469543 +0000 UTC m=+38.709857636"
Sep 17 20:49:58.323930 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:58.323895    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:49:58.324094 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:58.324022    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:49:58.324094 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:49:58.324078    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:50:02.324062557 +0000 UTC m=+43.220450635 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:49:58.709036 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:58.708828    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:58.941179 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:58.941130    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.132489 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:59.132396    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.382090 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:59.382059    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:49:59.488363 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:49:59.488336    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:01.498591 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:01.498558    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:01.572835 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:01.572790    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:02.353948 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:02.353914    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:02.354120 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:02.354071    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:50:02.354162 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:02.354136    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:50:10.354120529 +0000 UTC m=+51.250508617 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:50:07.028963 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.028897    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2dpzv" podStartSLOduration=10.570165661 podStartE2EDuration="13.028882257s" totalImagesPullingTime="2.458716596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:49:57.819797929 +0000 UTC m=+38.716186040" watchObservedRunningTime="2026-09-17 20:50:07.028882257 +0000 UTC m=+47.925270350"
Sep 17 20:50:07.029623 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.029604    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5995fd8d76-x6twc"]
Sep 17 20:50:07.075506 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.075467    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5995fd8d76-x6twc"]
Sep 17 20:50:07.075686 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.075603    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.078337 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078305    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078359    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078373    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078414    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078437    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078363    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 17 20:50:07.078499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.078467    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2ntl\""
Sep 17 20:50:07.182232 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.182196    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:07.188282 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.188255    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-default-certificate\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.188409 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.188289    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-stats-auth\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.188409 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.188384    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.188542 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.188466    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.188542 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.188495    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hmhm\" (UniqueName: \"kubernetes.io/projected/3fda01da-95e1-47b6-b6f2-ad05635edf2b-kube-api-access-7hmhm\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.283062 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.282985    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:07.289133 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.289102    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-default-certificate\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.289250 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.289142    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-stats-auth\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.289250 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.289203    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.289250 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.289227    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.289250 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.289246    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hmhm\" (UniqueName: \"kubernetes.io/projected/3fda01da-95e1-47b6-b6f2-ad05635edf2b-kube-api-access-7hmhm\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.289465 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.289442    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:07.289527 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.289515    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:07.789495544 +0000 UTC m=+48.685883636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:07.289638 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.289622    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:07.789607838 +0000 UTC m=+48.685995910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:07.293100 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.293070    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-stats-auth\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.293197 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.293108    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-default-certificate\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.317702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.317670    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hmhm\" (UniqueName: \"kubernetes.io/projected/3fda01da-95e1-47b6-b6f2-ad05635edf2b-kube-api-access-7hmhm\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.792561 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.792526    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.792727 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:07.792573    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:07.792727 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.792683    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:07.792800 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.792750    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:08.792737596 +0000 UTC m=+49.689125681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:07.792800 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:07.792766    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:08.79275986 +0000 UTC m=+49.689147932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:08.799998 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:08.799958    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:08.800393 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:08.800049    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:08.800393 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:08.800129    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:08.800393 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:08.800138    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:10.800120324 +0000 UTC m=+51.696508410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:08.800393 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:08.800175    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:10.800164334 +0000 UTC m=+51.696552421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:09.008051 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.008020    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"]
Sep 17 20:50:09.034481 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.034454    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.034835 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.034811    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"]
Sep 17 20:50:09.040162 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.040136    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 17 20:50:09.040297 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.040140    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 17 20:50:09.040297 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.040196    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 17 20:50:09.040452 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.040438    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:50:09.041371 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.041351    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9xr55\""
Sep 17 20:50:09.202509 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.202405    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbeaf482-76e0-4142-9f43-83b6e669c9c7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.202647 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.202543    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6zfg\" (UniqueName: \"kubernetes.io/projected/fbeaf482-76e0-4142-9f43-83b6e669c9c7-kube-api-access-f6zfg\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.202647 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.202596    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbeaf482-76e0-4142-9f43-83b6e669c9c7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.303394 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.303366    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6zfg\" (UniqueName: \"kubernetes.io/projected/fbeaf482-76e0-4142-9f43-83b6e669c9c7-kube-api-access-f6zfg\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.303492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.303409    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbeaf482-76e0-4142-9f43-83b6e669c9c7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.303492 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.303470    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbeaf482-76e0-4142-9f43-83b6e669c9c7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.304104 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.304080    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fbeaf482-76e0-4142-9f43-83b6e669c9c7-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.305620 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.305602    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fbeaf482-76e0-4142-9f43-83b6e669c9c7-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.312309 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.312280    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6zfg\" (UniqueName: \"kubernetes.io/projected/fbeaf482-76e0-4142-9f43-83b6e669c9c7-kube-api-access-f6zfg\") pod \"kube-storage-version-migrator-operator-55889f7b47-r8bgv\" (UID: \"fbeaf482-76e0-4142-9f43-83b6e669c9c7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.343269 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.343242    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"
Sep 17 20:50:09.628357 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.628327    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv"]
Sep 17 20:50:09.629243 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.629177    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:09.631149 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:09.631126    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbeaf482_76e0_4142_9f43_83b6e669c9c7.slice/crio-05fa58448fe00710fbed291578a1a94e345f423093ce6aff0fdd01a055274d8a WatchSource:0}: Error finding container 05fa58448fe00710fbed291578a1a94e345f423093ce6aff0fdd01a055274d8a: Status 404 returned error can't find the container with id 05fa58448fe00710fbed291578a1a94e345f423093ce6aff0fdd01a055274d8a
Sep 17 20:50:09.815129 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:09.815093    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv" event={"ID":"fbeaf482-76e0-4142-9f43-83b6e669c9c7","Type":"ContainerStarted","Data":"05fa58448fe00710fbed291578a1a94e345f423093ce6aff0fdd01a055274d8a"}
Sep 17 20:50:10.412688 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:10.412653    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:10.412858 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:10.412785    2587 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 17 20:50:10.412858 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:10.412847    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls podName:159ae6fc-327d-498f-b31e-94a7396a9abb nodeName:}" failed. No retries permitted until 2026-09-17 20:50:26.412830671 +0000 UTC m=+67.309218762 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls") pod "dns-default-brdvm" (UID: "159ae6fc-327d-498f-b31e-94a7396a9abb") : secret "dns-default-metrics-tls" not found
Sep 17 20:50:10.815050 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:10.815010    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:10.815253 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:10.815062    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:10.815253 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:10.815205    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:10.815253 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:10.815236    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:14.815215104 +0000 UTC m=+55.711603195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:10.815709 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:10.815272    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:14.815253951 +0000 UTC m=+55.711642027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:11.038361 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.038327    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"]
Sep 17 20:50:11.040077 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.040062    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.044374 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.044343    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pm6cf\""
Sep 17 20:50:11.044374 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.044358    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 17 20:50:11.044374 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.044366    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 17 20:50:11.044664 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.044649    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 17 20:50:11.045124 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.045104    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 17 20:50:11.055689 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.055663    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gfjlm"]
Sep 17 20:50:11.057565 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.057547    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.060083 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.060062    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 17 20:50:11.060208 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.060101    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 17 20:50:11.060208 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.060153    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 17 20:50:11.060298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.060268    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h4vct\""
Sep 17 20:50:11.060475 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.060462    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 17 20:50:11.067063 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.067011    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"]
Sep 17 20:50:11.074696 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.074670    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfjlm"]
Sep 17 20:50:11.077982 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.077954    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:11.117286 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117255    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd5781c3-0930-4d58-86b0-55ef0b39efe7-data-volume\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.117286 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117285    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5230ca67-439a-43e2-9c7c-a1523c7bba5b-serving-cert\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.117503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117305    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.117503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117351    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5230ca67-439a-43e2-9c7c-a1523c7bba5b-config\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.117503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117382    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsc56\" (UniqueName: \"kubernetes.io/projected/5230ca67-439a-43e2-9c7c-a1523c7bba5b-kube-api-access-zsc56\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.117503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117439    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.117503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117470    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfshq\" (UniqueName: \"kubernetes.io/projected/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-api-access-qfshq\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.117678 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.117521    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd5781c3-0930-4d58-86b0-55ef0b39efe7-crio-socket\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.147551 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.147521    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:11.218096 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218065    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd5781c3-0930-4d58-86b0-55ef0b39efe7-data-volume\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218096 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218098    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5230ca67-439a-43e2-9c7c-a1523c7bba5b-serving-cert\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.218322 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218121    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218322 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218140    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5230ca67-439a-43e2-9c7c-a1523c7bba5b-config\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.218322 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218243    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsc56\" (UniqueName: \"kubernetes.io/projected/5230ca67-439a-43e2-9c7c-a1523c7bba5b-kube-api-access-zsc56\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.218514 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218324    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218514 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.218415    2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:50:11.218514 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218443    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfshq\" (UniqueName: \"kubernetes.io/projected/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-api-access-qfshq\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218514 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.218506    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:11.718483561 +0000 UTC m=+52.614871653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : configmap "kube-rbac-proxy" not found
Sep 17 20:50:11.218729 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218590    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd5781c3-0930-4d58-86b0-55ef0b39efe7-crio-socket\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218729 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.218672    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:11.218821 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.218734    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:11.718711145 +0000 UTC m=+52.615099231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:11.218821 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218748    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dd5781c3-0930-4d58-86b0-55ef0b39efe7-data-volume\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218911 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218812    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dd5781c3-0930-4d58-86b0-55ef0b39efe7-crio-socket\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.218911 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.218871    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5230ca67-439a-43e2-9c7c-a1523c7bba5b-config\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.220621 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.220597    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5230ca67-439a-43e2-9c7c-a1523c7bba5b-serving-cert\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.229092 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.229067    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfshq\" (UniqueName: \"kubernetes.io/projected/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-api-access-qfshq\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.229213 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.229192    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsc56\" (UniqueName: \"kubernetes.io/projected/5230ca67-439a-43e2-9c7c-a1523c7bba5b-kube-api-access-zsc56\") pod \"service-ca-operator-57cdc4b768-mp87s\" (UID: \"5230ca67-439a-43e2-9c7c-a1523c7bba5b\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.351453 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.351359    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"
Sep 17 20:50:11.722513 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.722474    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.722701 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:11.722568    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:11.722701 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.722622    2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:50:11.722701 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.722698    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:12.722675669 +0000 UTC m=+53.619063783 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : configmap "kube-rbac-proxy" not found
Sep 17 20:50:11.722861 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.722751    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:11.722861 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:11.722815    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:12.722803803 +0000 UTC m=+53.619191891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:12.016966 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.016894    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s"]
Sep 17 20:50:12.019947 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:12.019924    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5230ca67_439a_43e2_9c7c_a1523c7bba5b.slice/crio-1b4752514e27cc65e0b0e05a823d26622e8331954ab01a8710a094ca07caefe5 WatchSource:0}: Error finding container 1b4752514e27cc65e0b0e05a823d26622e8331954ab01a8710a094ca07caefe5: Status 404 returned error can't find the container with id 1b4752514e27cc65e0b0e05a823d26622e8331954ab01a8710a094ca07caefe5
Sep 17 20:50:12.731965 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.731921    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:12.732146 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.732014    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:12.732146 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:12.732067    2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:50:12.732146 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:12.732139    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:12.732146 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:12.732148    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:14.732127155 +0000 UTC m=+55.628515232 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : configmap "kube-rbac-proxy" not found
Sep 17 20:50:12.732377 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:12.732191    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:14.73217553 +0000 UTC m=+55.628563606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:12.775898 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.775873    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9nksj_602783c1-0d44-4f67-ab95-e2c99cd0fe21/dns-node-resolver/0.log"
Sep 17 20:50:12.824083 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.824048    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv" event={"ID":"fbeaf482-76e0-4142-9f43-83b6e669c9c7","Type":"ContainerStarted","Data":"2530d0382bfa5b0e609a3f8ae8b85ea5fdbbc08602ce43ca95f7e9cb4e075e26"}
Sep 17 20:50:12.825345 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.825315    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s" event={"ID":"5230ca67-439a-43e2-9c7c-a1523c7bba5b","Type":"ContainerStarted","Data":"1b4752514e27cc65e0b0e05a823d26622e8331954ab01a8710a094ca07caefe5"}
Sep 17 20:50:12.840975 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:12.840915    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv" podStartSLOduration=2.687005798 podStartE2EDuration="4.84089407s" totalImagesPullingTime="2.153888272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:12.840082558 +0000 UTC m=+53.736470653" watchObservedRunningTime="2026-09-17 20:50:12.84089407 +0000 UTC m=+53.737282167"
Sep 17 20:50:13.473105 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.473072    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"]
Sep 17 20:50:13.476691 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.475712    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"
Sep 17 20:50:13.478736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.478712    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dssnl\""
Sep 17 20:50:13.478869 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.478849    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 17 20:50:13.478935 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.478894    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 17 20:50:13.482087 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.482062    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:13.485915 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.485885    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"]
Sep 17 20:50:13.538906 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.538872    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4rw7\" (UniqueName: \"kubernetes.io/projected/470a5f8b-7034-4af0-a46d-6cde6a4a492f-kube-api-access-p4rw7\") pod \"migrator-76794cbd6c-tlhb4\" (UID: \"470a5f8b-7034-4af0-a46d-6cde6a4a492f\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"
Sep 17 20:50:13.552142 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.552114    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:13.618165 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.618105    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:13.639441 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.639386    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4rw7\" (UniqueName: \"kubernetes.io/projected/470a5f8b-7034-4af0-a46d-6cde6a4a492f-kube-api-access-p4rw7\") pod \"migrator-76794cbd6c-tlhb4\" (UID: \"470a5f8b-7034-4af0-a46d-6cde6a4a492f\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"
Sep 17 20:50:13.647816 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.647789    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4rw7\" (UniqueName: \"kubernetes.io/projected/470a5f8b-7034-4af0-a46d-6cde6a4a492f-kube-api-access-p4rw7\") pod \"migrator-76794cbd6c-tlhb4\" (UID: \"470a5f8b-7034-4af0-a46d-6cde6a4a492f\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"
Sep 17 20:50:13.787984 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.787899    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"
Sep 17 20:50:13.977155 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:13.977130    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bn9d2_3c6387a1-2f78-4e35-a245-65dfe1c0dea5/node-ca/0.log"
Sep 17 20:50:14.054125 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.054054    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4"]
Sep 17 20:50:14.056827 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:14.056802    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod470a5f8b_7034_4af0_a46d_6cde6a4a492f.slice/crio-98930a41e257685c96bc514e0b7acc9079f6557ae9ee6efa39f73e463bf1e048 WatchSource:0}: Error finding container 98930a41e257685c96bc514e0b7acc9079f6557ae9ee6efa39f73e463bf1e048: Status 404 returned error can't find the container with id 98930a41e257685c96bc514e0b7acc9079f6557ae9ee6efa39f73e463bf1e048
Sep 17 20:50:14.576518 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.576486    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2dpzv_2c352448-d5b6-4c8f-aeba-ead56fa5b354/serve-healthcheck-canary/0.log"
Sep 17 20:50:14.749645 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.749613    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:14.749813 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.749685    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:14.749813 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.749742    2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:50:14.749890 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.749821    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:18.749801022 +0000 UTC m=+59.646189110 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : configmap "kube-rbac-proxy" not found
Sep 17 20:50:14.749890 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.749830    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:14.749890 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.749868    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:18.74985587 +0000 UTC m=+59.646243957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:14.833266 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.833178    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s" event={"ID":"5230ca67-439a-43e2-9c7c-a1523c7bba5b","Type":"ContainerStarted","Data":"c697941a209e12b3e9cf336f6fba7de67493d716a1b316f6f99ae4278c802126"}
Sep 17 20:50:14.835058 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.835024    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4" event={"ID":"470a5f8b-7034-4af0-a46d-6cde6a4a492f","Type":"ContainerStarted","Data":"98930a41e257685c96bc514e0b7acc9079f6557ae9ee6efa39f73e463bf1e048"}
Sep 17 20:50:14.850719 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.850687    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:14.850877 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:14.850745    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:14.850877 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.850862    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:14.850994 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.850890    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:22.850873338 +0000 UTC m=+63.747261425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:14.850994 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:14.850914    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:22.850899346 +0000 UTC m=+63.747287421 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:15.418508 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.418481    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:15.690308 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.690226    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:15.840041 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.839999    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4" event={"ID":"470a5f8b-7034-4af0-a46d-6cde6a4a492f","Type":"ContainerStarted","Data":"1607058c0fe8e6275d235d06cbc42438c8f9177e16b408e72836d20b3e4c0fe8"}
Sep 17 20:50:15.840220 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.840068    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4" event={"ID":"470a5f8b-7034-4af0-a46d-6cde6a4a492f","Type":"ContainerStarted","Data":"077658ef9f6ac972fda021b7bfd9c7f782f6e4a7a5507d7ceba2fbd59a1cfb18"}
Sep 17 20:50:15.856687 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.856636    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s" podStartSLOduration=2.263862491 podStartE2EDuration="4.85661563s" totalImagesPullingTime="2.592753139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:14.849600613 +0000 UTC m=+55.745988708" watchObservedRunningTime="2026-09-17 20:50:15.85661563 +0000 UTC m=+56.753003725"
Sep 17 20:50:15.856941 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.856909    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tlhb4" podStartSLOduration=1.5448768899999998 podStartE2EDuration="2.856904364s" totalImagesPullingTime="1.312027474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:15.856297937 +0000 UTC m=+56.752686031" watchObservedRunningTime="2026-09-17 20:50:15.856904364 +0000 UTC m=+56.753292483"
Sep 17 20:50:15.940358 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.940284    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:15.997288 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:15.997259    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:17.157620 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:17.157593    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:17.221934 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:17.221906    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:17.595600 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:17.595569    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:17.673022 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:17.672990    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:17.877733 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:17.877655    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:18.006237 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:18.006206    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:18.784066 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:18.784027    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:18.784501 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:18.784096    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:18.784501 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:18.784147    2587 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 17 20:50:18.784501 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:18.784222    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:26.784206432 +0000 UTC m=+67.680594505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : configmap "kube-rbac-proxy" not found
Sep 17 20:50:18.784501 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:18.784241    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:18.784501 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:18.784294    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:26.784281086 +0000 UTC m=+67.680669174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:19.507162 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:19.507133    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:19.574133 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:19.574104    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:19.764546 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:19.764475    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z9wf9"
Sep 17 20:50:19.930595 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:19.930546    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:20.007273 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:20.007246    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:20.473373 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:20.473342    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:22.915401 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:22.915360    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:22.915864 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:22.915531    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:22.915864 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:22.915563    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:38.91553654 +0000 UTC m=+79.811924629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : configmap references non-existent config key: service-ca.crt
Sep 17 20:50:22.915864 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:22.915627    2587 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 17 20:50:22.915864 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:22.915695    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs podName:3fda01da-95e1-47b6-b6f2-ad05635edf2b nodeName:}" failed. No retries permitted until 2026-09-17 20:50:38.915678553 +0000 UTC m=+79.812066642 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs") pod "router-default-5995fd8d76-x6twc" (UID: "3fda01da-95e1-47b6-b6f2-ad05635edf2b") : secret "router-metrics-certs-default" not found
Sep 17 20:50:22.986936 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:22.986894    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:23.242957 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:23.242924    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:23.340042 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:23.340011    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:23.444065 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:23.444033    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:23.581571 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:23.581491    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:23.687194 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:23.687159    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:24.161106 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:24.161074    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:24.220569 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:24.220539    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:24.308258 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:24.308228    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:24.875757 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:24.875728    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:25.279359 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.279327    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:25.336080 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.336044    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:50:25.336080 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.336080    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:50:25.339014 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.338996    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 17 20:50:25.339081 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.339053    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 17 20:50:25.346296 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:25.346273    2587 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 17 20:50:25.346381 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:25.346347    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs podName:ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa nodeName:}" failed. No retries permitted until 2026-09-17 20:51:29.346326996 +0000 UTC m=+130.242715074 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs") pod "network-metrics-daemon-ldldg" (UID: "ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa") : secret "metrics-daemon-secret" not found
Sep 17 20:50:25.348391 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.348374    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/658b0509-365d-4197-bb9e-682ffdccacb2-original-pull-secret\") pod \"global-pull-secret-syncer-fjsrk\" (UID: \"658b0509-365d-4197-bb9e-682ffdccacb2\") " pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:50:25.437149 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.437109    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:50:25.439922 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.439906    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 17 20:50:25.450883 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.450857    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 17 20:50:25.461293 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.461264    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5czz4\" (UniqueName: \"kubernetes.io/projected/d356f916-1ba5-463a-97ae-bc8030fc8d8f-kube-api-access-5czz4\") pod \"network-check-target-46x9r\" (UID: \"d356f916-1ba5-463a-97ae-bc8030fc8d8f\") " pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:50:25.568708 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.568625    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-j729c\""
Sep 17 20:50:25.571965 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.571946    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fjsrk"
Sep 17 20:50:25.576089 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.576069    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:50:25.678671 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.678640    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:25.781156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.781129    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:25.851080 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.850763    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-46x9r"]
Sep 17 20:50:25.855523 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.855499    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fjsrk"]
Sep 17 20:50:25.856106 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:25.856087    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd356f916_1ba5_463a_97ae_bc8030fc8d8f.slice/crio-de017985b4ca4f9504411ef64097efbcf8159a8d8d7ad502a5c42d4320960e52 WatchSource:0}: Error finding container de017985b4ca4f9504411ef64097efbcf8159a8d8d7ad502a5c42d4320960e52: Status 404 returned error can't find the container with id de017985b4ca4f9504411ef64097efbcf8159a8d8d7ad502a5c42d4320960e52
Sep 17 20:50:25.857232 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:25.857206    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658b0509_365d_4197_bb9e_682ffdccacb2.slice/crio-a886ad2d7443a4cb086d2cf3517a2a6edf54c80c669d385f69e7db8f7468e41c WatchSource:0}: Error finding container a886ad2d7443a4cb086d2cf3517a2a6edf54c80c669d385f69e7db8f7468e41c: Status 404 returned error can't find the container with id a886ad2d7443a4cb086d2cf3517a2a6edf54c80c669d385f69e7db8f7468e41c
Sep 17 20:50:25.865104 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.865077    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-46x9r" event={"ID":"d356f916-1ba5-463a-97ae-bc8030fc8d8f","Type":"ContainerStarted","Data":"de017985b4ca4f9504411ef64097efbcf8159a8d8d7ad502a5c42d4320960e52"}
Sep 17 20:50:25.866179 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.866161    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fjsrk" event={"ID":"658b0509-365d-4197-bb9e-682ffdccacb2","Type":"ContainerStarted","Data":"a886ad2d7443a4cb086d2cf3517a2a6edf54c80c669d385f69e7db8f7468e41c"}
Sep 17 20:50:25.881517 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:25.881493    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:26.448402 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.448363    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:26.450937 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.450913    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/159ae6fc-327d-498f-b31e-94a7396a9abb-metrics-tls\") pod \"dns-default-brdvm\" (UID: \"159ae6fc-327d-498f-b31e-94a7396a9abb\") " pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:26.496571 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.496504    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:26.647214 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.647179    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h6qzc\""
Sep 17 20:50:26.655205 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.654986    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:26.686880 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.686849    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:26.852702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.852309    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:26.852702 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.852453    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:26.852702 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:26.852487    2587 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:26.852702 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:26.852574    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls podName:dd5781c3-0930-4d58-86b0-55ef0b39efe7 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:42.852552419 +0000 UTC m=+83.748940494 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gfjlm" (UID: "dd5781c3-0930-4d58-86b0-55ef0b39efe7") : secret "insights-runtime-extractor-tls" not found
Sep 17 20:50:26.853107 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.853069    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dd5781c3-0930-4d58-86b0-55ef0b39efe7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:26.891818 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.891784    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:26.948860 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.948827    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-brdvm"]
Sep 17 20:50:26.953797 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:26.953768    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159ae6fc_327d_498f_b31e_94a7396a9abb.slice/crio-ca9baf905a822c3ad0ebde793b280912716190480fe3a23a3718cb17bfd598a5 WatchSource:0}: Error finding container ca9baf905a822c3ad0ebde793b280912716190480fe3a23a3718cb17bfd598a5: Status 404 returned error can't find the container with id ca9baf905a822c3ad0ebde793b280912716190480fe3a23a3718cb17bfd598a5
Sep 17 20:50:26.968525 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:26.968446    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:27.245783 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:27.245745    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:27.346118 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:27.346066    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:27.475905 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:27.475866    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:27.873992 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:27.873951    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brdvm" event={"ID":"159ae6fc-327d-498f-b31e-94a7396a9abb","Type":"ContainerStarted","Data":"ca9baf905a822c3ad0ebde793b280912716190480fe3a23a3718cb17bfd598a5"}
Sep 17 20:50:27.887147 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:27.887031    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:28.484067 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:28.484022    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:28.884332 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:28.884254    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:29.314841 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:29.314790    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:29.417614 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:29.417581    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:29.515539 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:29.515511    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:29.878409 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:29.878354    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:30.278744 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:30.278705    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:31.075144 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.075114    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:31.277378 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.277296    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:31.481579 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.481548    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:31.876969 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.876940    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:31.888211 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.888180    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fjsrk" event={"ID":"658b0509-365d-4197-bb9e-682ffdccacb2","Type":"ContainerStarted","Data":"af1ad55c913064a5d43e34c67f3ebaa041e95654b7edc4d3a39de9dc37b88c36"}
Sep 17 20:50:31.889732 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.889701    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-46x9r" event={"ID":"d356f916-1ba5-463a-97ae-bc8030fc8d8f","Type":"ContainerStarted","Data":"4f5ca2362d45701ff86a813ef1b20e71b4d61c2b0da4e32e09347dd7438b6985"}
Sep 17 20:50:31.889857 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.889800    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:50:31.891372 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.891351    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brdvm" event={"ID":"159ae6fc-327d-498f-b31e-94a7396a9abb","Type":"ContainerStarted","Data":"81a3653c5de2cb38c7b2ff9dcd609b010bc3ca6df2c6a9962cd3d6482de1722d"}
Sep 17 20:50:31.891472 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.891393    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-brdvm" event={"ID":"159ae6fc-327d-498f-b31e-94a7396a9abb","Type":"ContainerStarted","Data":"e01b77b99e4d4e6b5232abef78437e77617eb044ec20a53070ff8550f53745a8"}
Sep 17 20:50:31.891472 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.891467    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:31.903970 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.903922    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fjsrk" podStartSLOduration=66.734919901 podStartE2EDuration="1m11.903908756s" totalImagesPullingTime="5.168988855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:31.903293764 +0000 UTC m=+72.799681859" watchObservedRunningTime="2026-09-17 20:50:31.903908756 +0000 UTC m=+72.800296884"
Sep 17 20:50:31.921346 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.921302    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-brdvm" podStartSLOduration=33.860384991 podStartE2EDuration="37.921284598s" totalImagesPullingTime="4.060899607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:31.920576017 +0000 UTC m=+72.816964111" watchObservedRunningTime="2026-09-17 20:50:31.921284598 +0000 UTC m=+72.817672694"
Sep 17 20:50:31.936013 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:31.935972    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-46x9r" podStartSLOduration=66.781101461 podStartE2EDuration="1m11.935958511s" totalImagesPullingTime="5.15485705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:31.935247428 +0000 UTC m=+72.831635523" watchObservedRunningTime="2026-09-17 20:50:31.935958511 +0000 UTC m=+72.832346604"
Sep 17 20:50:32.475950 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:32.475918    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:32.875374 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:32.875288    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:33.275219 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:33.275182    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:33.480791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:33.480748    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:33.704910 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:33.704816    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:34.152205 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:34.152174    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:34.252242 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:34.252211    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:34.465651 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:34.465622    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:34.687785 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:34.687758    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:35.039970 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:35.039939    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:35.124225 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:35.124198    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:35.853481 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:35.853454    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:36.052784 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:36.052754    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:36.153835 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:36.153736    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:36.254974 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:36.254944    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:37.025603 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:37.025572    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:38.949105 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:38.949068    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:38.949105 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:38.949103    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:38.949735 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:38.949717    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fda01da-95e1-47b6-b6f2-ad05635edf2b-service-ca-bundle\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:38.951483 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:38.951463    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3fda01da-95e1-47b6-b6f2-ad05635edf2b-metrics-certs\") pod \"router-default-5995fd8d76-x6twc\" (UID: \"3fda01da-95e1-47b6-b6f2-ad05635edf2b\") " pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:39.187397 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.187367    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2ntl\""
Sep 17 20:50:39.195573 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.195551    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:39.484900 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.484870    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5995fd8d76-x6twc"]
Sep 17 20:50:39.487943 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:39.487918    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fda01da_95e1_47b6_b6f2_ad05635edf2b.slice/crio-3b1f1824c8a9cf33ec955412d36528c5427135e7227506c53d39182d35628670 WatchSource:0}: Error finding container 3b1f1824c8a9cf33ec955412d36528c5427135e7227506c53d39182d35628670: Status 404 returned error can't find the container with id 3b1f1824c8a9cf33ec955412d36528c5427135e7227506c53d39182d35628670
Sep 17 20:50:39.917891 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.917802    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5995fd8d76-x6twc" event={"ID":"3fda01da-95e1-47b6-b6f2-ad05635edf2b","Type":"ContainerStarted","Data":"617346ae6c1a23a5a78a044be6c0a9e7103bb678c18fae99276f05f7809f08cc"}
Sep 17 20:50:39.917891 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.917855    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5995fd8d76-x6twc" event={"ID":"3fda01da-95e1-47b6-b6f2-ad05635edf2b","Type":"ContainerStarted","Data":"3b1f1824c8a9cf33ec955412d36528c5427135e7227506c53d39182d35628670"}
Sep 17 20:50:39.952305 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:39.952256    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5995fd8d76-x6twc" podStartSLOduration=32.952241455 podStartE2EDuration="32.952241455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:39.951353987 +0000 UTC m=+80.847742080" watchObservedRunningTime="2026-09-17 20:50:39.952241455 +0000 UTC m=+80.848629550"
Sep 17 20:50:40.196389 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:40.196305    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:40.198813 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:40.198788    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:40.921143 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:40.921103    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:40.922299 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:40.922278    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5995fd8d76-x6twc"
Sep 17 20:50:41.897539 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:41.897416    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-brdvm"
Sep 17 20:50:41.955028 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:41.954994    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:42.087282 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.087253    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:42.301021 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.300989    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:42.392105 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.392068    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:42.705791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.705711    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:42.876745 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.876705    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:42.878998 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:42.878979    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dd5781c3-0930-4d58-86b0-55ef0b39efe7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gfjlm\" (UID: \"dd5781c3-0930-4d58-86b0-55ef0b39efe7\") " pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:43.170866 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.170833    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h4vct\""
Sep 17 20:50:43.177166 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.177147    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gfjlm"
Sep 17 20:50:43.478610 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.478548    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gfjlm"]
Sep 17 20:50:43.481074 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:43.481048    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd5781c3_0930_4d58_86b0_55ef0b39efe7.slice/crio-f51a44e24ddb3cc0232beb72560b2ce8e6bc2b6f643ff29bbd3d15a3f0a99e79 WatchSource:0}: Error finding container f51a44e24ddb3cc0232beb72560b2ce8e6bc2b6f643ff29bbd3d15a3f0a99e79: Status 404 returned error can't find the container with id f51a44e24ddb3cc0232beb72560b2ce8e6bc2b6f643ff29bbd3d15a3f0a99e79
Sep 17 20:50:43.668330 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.668296    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"]
Sep 17 20:50:43.671524 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.671500    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.678603 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.678573    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 17 20:50:43.678745 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.678728    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 17 20:50:43.678996 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.678981    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 17 20:50:43.679053 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.679039    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 17 20:50:43.685133 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.685115    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 17 20:50:43.699315 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.699297    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 17 20:50:43.700268 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.700252    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 17 20:50:43.783238 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783151    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.783238 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783195    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kkcc\" (UniqueName: \"kubernetes.io/projected/555866f3-6e4f-42ca-b2e2-67574bb728a3-kube-api-access-7kkcc\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.783238 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783221    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.783479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783318    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/555866f3-6e4f-42ca-b2e2-67574bb728a3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.783479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783364    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.783479 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.783391    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.859726 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.859690    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"]
Sep 17 20:50:43.884238 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884212    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/555866f3-6e4f-42ca-b2e2-67574bb728a3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.884355 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884250    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.884355 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884268    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.884355 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884291    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.884355 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884318    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kkcc\" (UniqueName: \"kubernetes.io/projected/555866f3-6e4f-42ca-b2e2-67574bb728a3-kube-api-access-7kkcc\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.884570 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.884448    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.885139 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.885112    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/555866f3-6e4f-42ca-b2e2-67574bb728a3-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.886925 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.886897    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.886925 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.886906    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-ca\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.887048 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.886982    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.887048 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.886978    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/555866f3-6e4f-42ca-b2e2-67574bb728a3-hub\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.931617 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.931589    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfjlm" event={"ID":"dd5781c3-0930-4d58-86b0-55ef0b39efe7","Type":"ContainerStarted","Data":"6d527a395b668cb211c8a913da5405748aee4e3bbdc34ff53fa631c0145af332"}
Sep 17 20:50:43.931765 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.931640    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfjlm" event={"ID":"dd5781c3-0930-4d58-86b0-55ef0b39efe7","Type":"ContainerStarted","Data":"f51a44e24ddb3cc0232beb72560b2ce8e6bc2b6f643ff29bbd3d15a3f0a99e79"}
Sep 17 20:50:43.932545 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.932527    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kkcc\" (UniqueName: \"kubernetes.io/projected/555866f3-6e4f-42ca-b2e2-67574bb728a3-kube-api-access-7kkcc\") pod \"cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4\" (UID: \"555866f3-6e4f-42ca-b2e2-67574bb728a3\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:43.993064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:43.993034    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"
Sep 17 20:50:44.113927 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.113888    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:44.275766 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.275736    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:44.296894 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.296830    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4"]
Sep 17 20:50:44.300056 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:44.300033    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod555866f3_6e4f_42ca_b2e2_67574bb728a3.slice/crio-a17201396e79bde63dc4204a415c0a3a336e6ec36bab9d1744def22efc5ec68c WatchSource:0}: Error finding container a17201396e79bde63dc4204a415c0a3a336e6ec36bab9d1744def22efc5ec68c: Status 404 returned error can't find the container with id a17201396e79bde63dc4204a415c0a3a336e6ec36bab9d1744def22efc5ec68c
Sep 17 20:50:44.429080 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.429051    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:44.563488 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.563393    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:44.936569 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.936476    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4" event={"ID":"555866f3-6e4f-42ca-b2e2-67574bb728a3","Type":"ContainerStarted","Data":"a17201396e79bde63dc4204a415c0a3a336e6ec36bab9d1744def22efc5ec68c"}
Sep 17 20:50:44.938516 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:44.938488    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfjlm" event={"ID":"dd5781c3-0930-4d58-86b0-55ef0b39efe7","Type":"ContainerStarted","Data":"eb131be57855201ffe38ecec841fe98bd264dbc58fac61fa42344c823cba41a9"}
Sep 17 20:50:45.944637 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:45.944545    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gfjlm" event={"ID":"dd5781c3-0930-4d58-86b0-55ef0b39efe7","Type":"ContainerStarted","Data":"3fe2cc808cde85d800f33739b8400b52cb80c4335989b26fb78c98213ba9fab3"}
Sep 17 20:50:45.974150 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:45.974097    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gfjlm" podStartSLOduration=32.869532498 podStartE2EDuration="34.974080283s" totalImagesPullingTime="2.104547785s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 20:50:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:50:45.972519425 +0000 UTC m=+86.868907515" watchObservedRunningTime="2026-09-17 20:50:45.974080283 +0000 UTC m=+86.870468377"
Sep 17 20:50:51.110046 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:51.110010    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:51.213738 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:51.213704    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:51.711470 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:51.711416    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:51.811813 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:51.811775    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:52.183311 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:52.183224    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:52.291789 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:52.291758    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:52.696070 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:52.696028    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:52.797438 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:52.797403    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:52.867149 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:52.867123    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:54.890353 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:54.890319    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:54.956279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:54.956253    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:55.291858 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:55.291823    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.339065 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.339031    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.555189 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.555154    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.603282 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.603206    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-spldr"]
Sep 17 20:50:57.609392 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.609363    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.609913 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.609791    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mzph6"]
Sep 17 20:50:57.612367 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.612341    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 17 20:50:57.613256 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.613236    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.613723 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.613693    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 17 20:50:57.615450 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.614736    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5h5j9\""
Sep 17 20:50:57.615450 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.615030    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 17 20:50:57.615722 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.615700    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 17 20:50:57.616198 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.616171    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 17 20:50:57.616522 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.616503    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 17 20:50:57.617092 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.617071    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 17 20:50:57.617221 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.617083    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vfzf5\""
Sep 17 20:50:57.617757 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.617734    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 17 20:50:57.624039 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.623950    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 17 20:50:57.624301 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.624276    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-spldr"]
Sep 17 20:50:57.650799 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.650746    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.692879 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.692794    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-textfile\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.692879 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.692840    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.692891    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0f8ddb33-fb5f-4238-974e-2de5e4847d09-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.692917    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.692966    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-metrics-client-ca\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693026    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsj78\" (UniqueName: \"kubernetes.io/projected/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-api-access-hsj78\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693057    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v4n6\" (UniqueName: \"kubernetes.io/projected/6d2724f0-7a6f-481a-af6a-91ab603599d8-kube-api-access-4v4n6\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693083    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-wtmp\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693156 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693156    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693186    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693215    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693238    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-sys\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693263    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-root\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693285    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.693557 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.693331    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.719711 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.719676    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.794312 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794267    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsj78\" (UniqueName: \"kubernetes.io/projected/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-api-access-hsj78\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.794312 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794316    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v4n6\" (UniqueName: \"kubernetes.io/projected/6d2724f0-7a6f-481a-af6a-91ab603599d8-kube-api-access-4v4n6\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794335    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-wtmp\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794361    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794387    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794416    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794461    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-sys\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794487    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-root\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794508    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794554 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794549    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.794856 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794583    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-textfile\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794856 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794618    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.794856 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794670    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0f8ddb33-fb5f-4238-974e-2de5e4847d09-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.794856 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:57.794746    2587 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 17 20:50:57.794856 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:57.794799    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls podName:6d2724f0-7a6f-481a-af6a-91ab603599d8 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:58.294784379 +0000 UTC m=+99.191172452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls") pod "node-exporter-mzph6" (UID: "6d2724f0-7a6f-481a-af6a-91ab603599d8") : secret "node-exporter-tls" not found
Sep 17 20:50:57.795032 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.794975    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0f8ddb33-fb5f-4238-974e-2de5e4847d09-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.795074 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795034    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-sys\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.795074 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795042    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-root\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795178    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-wtmp\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795214    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795220    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795301    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-metrics-client-ca\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795044    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795485    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-textfile\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:57.795642    2587 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:50:57.795715    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls podName:0f8ddb33-fb5f-4238-974e-2de5e4847d09 nodeName:}" failed. No retries permitted until 2026-09-17 20:50:58.295698285 +0000 UTC m=+99.192086362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-spldr" (UID: "0f8ddb33-fb5f-4238-974e-2de5e4847d09") : secret "kube-state-metrics-tls" not found
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795841    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6d2724f0-7a6f-481a-af6a-91ab603599d8-metrics-client-ca\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.796064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.795977    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0f8ddb33-fb5f-4238-974e-2de5e4847d09-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.799201 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.799111    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.800190 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.800162    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.803927 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.803893    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsj78\" (UniqueName: \"kubernetes.io/projected/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-api-access-hsj78\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:57.807406 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.807381    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v4n6\" (UniqueName: \"kubernetes.io/projected/6d2724f0-7a6f-481a-af6a-91ab603599d8-kube-api-access-4v4n6\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:57.820357 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.820273    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:57.921120 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:57.921036    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:58.023904 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.023870    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:58.120864 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.120827    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:58.221335 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.221298    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:58.299456 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.299413    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:58.299630 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.299472    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:58.302071 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.302035    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6d2724f0-7a6f-481a-af6a-91ab603599d8-node-exporter-tls\") pod \"node-exporter-mzph6\" (UID: \"6d2724f0-7a6f-481a-af6a-91ab603599d8\") " pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:58.302204 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.302173    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0f8ddb33-fb5f-4238-974e-2de5e4847d09-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-spldr\" (UID: \"0f8ddb33-fb5f-4238-974e-2de5e4847d09\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:58.526996 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.526916    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr"
Sep 17 20:50:58.534064 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.534039    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzph6"
Sep 17 20:50:58.602801 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.602767    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:58.682101 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:58.682004    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:59.230364 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.230331    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:50:59.319161 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:59.319132    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d2724f0_7a6f_481a_af6a_91ab603599d8.slice/crio-a49cc4a635529142896906a34a068d958724ffa3a0504b29ca528019bc75cd14 WatchSource:0}: Error finding container a49cc4a635529142896906a34a068d958724ffa3a0504b29ca528019bc75cd14: Status 404 returned error can't find the container with id a49cc4a635529142896906a34a068d958724ffa3a0504b29ca528019bc75cd14
Sep 17 20:50:59.588840 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.588810    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-spldr"]
Sep 17 20:50:59.591478 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:50:59.591443    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8ddb33_fb5f_4238_974e_2de5e4847d09.slice/crio-4be579cfce16e68d57a7310deed852d8263e77a27274abf38c4b75b2241031bb WatchSource:0}: Error finding container 4be579cfce16e68d57a7310deed852d8263e77a27274abf38c4b75b2241031bb: Status 404 returned error can't find the container with id 4be579cfce16e68d57a7310deed852d8263e77a27274abf38c4b75b2241031bb
Sep 17 20:50:59.994180 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.994147    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzph6" event={"ID":"6d2724f0-7a6f-481a-af6a-91ab603599d8","Type":"ContainerStarted","Data":"a49cc4a635529142896906a34a068d958724ffa3a0504b29ca528019bc75cd14"}
Sep 17 20:50:59.995373 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.995348    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr" event={"ID":"0f8ddb33-fb5f-4238-974e-2de5e4847d09","Type":"ContainerStarted","Data":"4be579cfce16e68d57a7310deed852d8263e77a27274abf38c4b75b2241031bb"}
Sep 17 20:50:59.997325 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.997300    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4" event={"ID":"555866f3-6e4f-42ca-b2e2-67574bb728a3","Type":"ContainerStarted","Data":"33de1b1020a4d36db907ce4bb637a1d67675099dbad785cc6d9f4d2f21b1161e"}
Sep 17 20:50:59.997457 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.997337    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4" event={"ID":"555866f3-6e4f-42ca-b2e2-67574bb728a3","Type":"ContainerStarted","Data":"a8b66b0f91b982acb992c9fa82d29227f99ba44267c7cc26e41cd681299a0f02"}
Sep 17 20:50:59.997457 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:50:59.997355    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4" event={"ID":"555866f3-6e4f-42ca-b2e2-67574bb728a3","Type":"ContainerStarted","Data":"c78e32e1e9074dd40c1626519ccaf4953662c9450d8a790622b36309ddc7b6dc"}
Sep 17 20:51:00.018767 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:00.018707    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcdccf6fc-ljrc4" podStartSLOduration=1.930693287 podStartE2EDuration="17.018681111s" totalImagesPullingTime="15.087987824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:00.01785971 +0000 UTC m=+100.914247805" watchObservedRunningTime="2026-09-17 20:51:00.018681111 +0000 UTC m=+100.915069211"
Sep 17 20:51:00.497188 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:00.497149    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:00.607322 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:00.607233    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:00.968772 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:00.968693    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:01.003728 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.003701    2587 generic.go:356] "Generic (PLEG): container finished" podID="6d2724f0-7a6f-481a-af6a-91ab603599d8" containerID="55028f3cf715abf4fdfa1d81ac341e23269a1e56dab35c9d85c74b9bc9516bd8" exitCode=0
Sep 17 20:51:01.003855 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.003797    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzph6" event={"ID":"6d2724f0-7a6f-481a-af6a-91ab603599d8","Type":"ContainerDied","Data":"55028f3cf715abf4fdfa1d81ac341e23269a1e56dab35c9d85c74b9bc9516bd8"}
Sep 17 20:51:01.007292 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.007174    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr" event={"ID":"0f8ddb33-fb5f-4238-974e-2de5e4847d09","Type":"ContainerStarted","Data":"775e5861d0c832a9b024afbe8ca4e7b00cd1e1986dead38d609d9cd6b4b150c5"}
Sep 17 20:51:01.007292 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.007238    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr" event={"ID":"0f8ddb33-fb5f-4238-974e-2de5e4847d09","Type":"ContainerStarted","Data":"c5d43418a4d3b8eed526cbef2ba841c1cbf6a297d8f2d60785375b8fc68382a1"}
Sep 17 20:51:01.039791 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.039761    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:01.909339 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.909303    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7647ffd697-gjb2q"]
Sep 17 20:51:01.912477 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.912458    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:01.917505 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917474    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 17 20:51:01.917615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917506    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-c49sn\""
Sep 17 20:51:01.917615 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917590    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bfhvec1lrdhs4\""
Sep 17 20:51:01.917706 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917620    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 17 20:51:01.917706 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917697    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 17 20:51:01.917917 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.917902    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 17 20:51:01.924249 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:01.924230    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7647ffd697-gjb2q"]
Sep 17 20:51:02.013276 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.013235    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzph6" event={"ID":"6d2724f0-7a6f-481a-af6a-91ab603599d8","Type":"ContainerStarted","Data":"774bd8f9e042bb37743cba02b20b19ce762705c8df2d5848b3459caabce14120"}
Sep 17 20:51:02.013461 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.013297    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzph6" event={"ID":"6d2724f0-7a6f-481a-af6a-91ab603599d8","Type":"ContainerStarted","Data":"8a6e6bed88208e187dd197d7dfbe43b0ab099016b7475fd1aec2e1455c495d2a"}
Sep 17 20:51:02.015244 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.015216    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr" event={"ID":"0f8ddb33-fb5f-4238-974e-2de5e4847d09","Type":"ContainerStarted","Data":"0eea4c784cf415ab59371c919082e33fe1803a73ae25971c6bb0c1a9317ccf8c"}
Sep 17 20:51:02.030205 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030178    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-tls\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030337 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030207    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030337 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030232    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-metrics-server-audit-profiles\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030337 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030252    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-client-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030372    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-255jb\" (UniqueName: \"kubernetes.io/projected/b9c95342-2ca7-4453-ab3d-44b73479128e-kube-api-access-255jb\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030470    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b9c95342-2ca7-4453-ab3d-44b73479128e-audit-log\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.030503 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.030492    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-client-certs\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.046988 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.046945    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mzph6" podStartSLOduration=4.041727271 podStartE2EDuration="5.046930182s" totalImagesPullingTime="932.360398ms" totalInitContainerRuntime="72.842513ms" isStatefulPod=true podCreationTimestamp="2026-09-17 20:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:02.046052306 +0000 UTC m=+102.942440397" watchObservedRunningTime="2026-09-17 20:51:02.046930182 +0000 UTC m=+102.943318277"
Sep 17 20:51:02.072555 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.072496    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-spldr" podStartSLOduration=3.809227411 podStartE2EDuration="5.072480704s" totalImagesPullingTime="1.263253293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:50:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:02.071835597 +0000 UTC m=+102.968223691" watchObservedRunningTime="2026-09-17 20:51:02.072480704 +0000 UTC m=+102.968868798"
Sep 17 20:51:02.111161 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.111132    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:02.131344 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131312    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-tls\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131508 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131351    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131508 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131484    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-metrics-server-audit-profiles\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131627 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131521    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-client-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131627 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131575    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-255jb\" (UniqueName: \"kubernetes.io/projected/b9c95342-2ca7-4453-ab3d-44b73479128e-kube-api-access-255jb\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131664    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b9c95342-2ca7-4453-ab3d-44b73479128e-audit-log\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.131721 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131705    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-client-certs\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.132401 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.131993    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b9c95342-2ca7-4453-ab3d-44b73479128e-audit-log\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.132772 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.132748    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.132864 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.132809    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b9c95342-2ca7-4453-ab3d-44b73479128e-metrics-server-audit-profiles\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.133925 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.133904    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-client-ca-bundle\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.134033 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.134017    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-tls\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.134221 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.134199    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b9c95342-2ca7-4453-ab3d-44b73479128e-secret-metrics-server-client-certs\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.150369 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.150342    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-255jb\" (UniqueName: \"kubernetes.io/projected/b9c95342-2ca7-4453-ab3d-44b73479128e-kube-api-access-255jb\") pod \"metrics-server-7647ffd697-gjb2q\" (UID: \"b9c95342-2ca7-4453-ab3d-44b73479128e\") " pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.221732 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.221698    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:02.244515 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.244486    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:02.428736 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.428702    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:02.494112 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.494037    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7647ffd697-gjb2q"]
Sep 17 20:51:02.501061 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:51:02.501033    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c95342_2ca7_4453_ab3d_44b73479128e.slice/crio-2cf085a60f9b6fb640e4a95333e15bed53cf3c74bd51e1865d236e646b455f7d WatchSource:0}: Error finding container 2cf085a60f9b6fb640e4a95333e15bed53cf3c74bd51e1865d236e646b455f7d: Status 404 returned error can't find the container with id 2cf085a60f9b6fb640e4a95333e15bed53cf3c74bd51e1865d236e646b455f7d
Sep 17 20:51:02.520612 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.520581    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:02.648871 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.648838    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:02.897903 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:02.897831    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-46x9r"
Sep 17 20:51:03.019686 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:03.019643    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q" event={"ID":"b9c95342-2ca7-4453-ab3d-44b73479128e","Type":"ContainerStarted","Data":"2cf085a60f9b6fb640e4a95333e15bed53cf3c74bd51e1865d236e646b455f7d"}
Sep 17 20:51:03.792657 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:03.792604    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:03.922858 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:03.922820    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:04.024822 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:04.024791    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:04.953251 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:04.953219    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:05.028472 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:05.028436    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q" event={"ID":"b9c95342-2ca7-4453-ab3d-44b73479128e","Type":"ContainerStarted","Data":"9f7e11a3aed054405e03d7bab4b4a24f5ea55f86a1ab048b3b275b480eabed7f"}
Sep 17 20:51:05.048900 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:05.048844    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q" podStartSLOduration=2.228258834 podStartE2EDuration="4.048829411s" totalImagesPullingTime="1.820570577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:51:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:05.04779483 +0000 UTC m=+105.944182925" watchObservedRunningTime="2026-09-17 20:51:05.048829411 +0000 UTC m=+105.945217506"
Sep 17 20:51:05.051259 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:05.051221    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:06.815886 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:06.815834    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:06.874658 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:06.874620    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:10.172005 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:10.171964    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:10.288621 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:10.288590    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.083462 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.083412    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.185335 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.185291    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.248516 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.248481    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.379240 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.379159    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.476256 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.476222    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.773047 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.773015    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:14.886674 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:14.886640    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:16.687767 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:16.687737    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:16.747929 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:16.747897    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:22.086002 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:22.085967    2587 generic.go:356] "Generic (PLEG): container finished" podID="5230ca67-439a-43e2-9c7c-a1523c7bba5b" containerID="c697941a209e12b3e9cf336f6fba7de67493d716a1b316f6f99ae4278c802126" exitCode=0
Sep 17 20:51:22.086393 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:22.086041    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s" event={"ID":"5230ca67-439a-43e2-9c7c-a1523c7bba5b","Type":"ContainerDied","Data":"c697941a209e12b3e9cf336f6fba7de67493d716a1b316f6f99ae4278c802126"}
Sep 17 20:51:22.086393 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:22.086339    2587 scope.go:117] "RemoveContainer" containerID="c697941a209e12b3e9cf336f6fba7de67493d716a1b316f6f99ae4278c802126"
Sep 17 20:51:22.221938 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:22.221907    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:22.222091 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:22.221994    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:23.091219 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:23.091178    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mp87s" event={"ID":"5230ca67-439a-43e2-9c7c-a1523c7bba5b","Type":"ContainerStarted","Data":"fdd1cf799597db874305fd632c2763ca1496df1b87631530e93f5e925835e3a6"}
Sep 17 20:51:25.058014 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:25.057978    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:25.158530 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:25.158471    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:25.262401 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:25.262365    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:26.908696 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:26.908658    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.022133 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:27.022098    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.139725 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:27.139687    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:27.236523 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:27.236493    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:28.110379 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:28.110346    2587 generic.go:356] "Generic (PLEG): container finished" podID="fbeaf482-76e0-4142-9f43-83b6e669c9c7" containerID="2530d0382bfa5b0e609a3f8ae8b85ea5fdbbc08602ce43ca95f7e9cb4e075e26" exitCode=0
Sep 17 20:51:28.110828 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:28.110390    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv" event={"ID":"fbeaf482-76e0-4142-9f43-83b6e669c9c7","Type":"ContainerDied","Data":"2530d0382bfa5b0e609a3f8ae8b85ea5fdbbc08602ce43ca95f7e9cb4e075e26"}
Sep 17 20:51:28.110828 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:28.110686    2587 scope.go:117] "RemoveContainer" containerID="2530d0382bfa5b0e609a3f8ae8b85ea5fdbbc08602ce43ca95f7e9cb4e075e26"
Sep 17 20:51:29.115770 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.115738    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-r8bgv" event={"ID":"fbeaf482-76e0-4142-9f43-83b6e669c9c7","Type":"ContainerStarted","Data":"11dc18e06b0630942af167a666b5ced65130c494beb48cb0295f05f41080076d"}
Sep 17 20:51:29.378730 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.378631    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.444302 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.444266    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:51:29.446682 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.446644    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa-metrics-certs\") pod \"network-metrics-daemon-ldldg\" (UID: \"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa\") " pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:51:29.459221 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.459191    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:29.478792 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.478763    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-775ms\""
Sep 17 20:51:29.487024 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.487001    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldldg"
Sep 17 20:51:29.802715 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:29.802671    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldldg"]
Sep 17 20:51:29.808017 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:51:29.807989    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac1cee9d_3a8b_4c03_ab99_f2f5634f02aa.slice/crio-c2b904e2c15bac0ee3334052c0d453c56aaf4ba1492d01766450ee4469a011e7 WatchSource:0}: Error finding container c2b904e2c15bac0ee3334052c0d453c56aaf4ba1492d01766450ee4469a011e7: Status 404 returned error can't find the container with id c2b904e2c15bac0ee3334052c0d453c56aaf4ba1492d01766450ee4469a011e7
Sep 17 20:51:30.120116 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:30.120033    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldldg" event={"ID":"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa","Type":"ContainerStarted","Data":"c2b904e2c15bac0ee3334052c0d453c56aaf4ba1492d01766450ee4469a011e7"}
Sep 17 20:51:32.128342 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:32.128311    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldldg" event={"ID":"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa","Type":"ContainerStarted","Data":"08bcc2f9a292162a5ca43b0193d3099eac89b78894e12c01ac2e941deaeb1489"}
Sep 17 20:51:32.128803 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:32.128362    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldldg" event={"ID":"ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa","Type":"ContainerStarted","Data":"13b079120bb81c1ee6c99d582b46a449b056fe7b6a90ce4fb1122605e7b8d65a"}
Sep 17 20:51:32.158527 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:32.158454    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ldldg" podStartSLOduration=130.771851573 podStartE2EDuration="2m12.158408599s" totalImagesPullingTime="1.386557026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:51:32.157168077 +0000 UTC m=+133.053556173" watchObservedRunningTime="2026-09-17 20:51:32.158408599 +0000 UTC m=+133.054796694"
Sep 17 20:51:32.276473 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:32.276443    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:32.343149 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:32.343121    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.131655 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:35.131626    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:35.232589 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:35.232543    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:51:42.228470 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:42.228414    2587 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:51:42.233226 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:51:42.233200    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7647ffd697-gjb2q"
Sep 17 20:52:13.655670 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:13.655591    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:13.756366 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:13.756335    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:25.600873 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:25.600836    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:25.704799 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:25.704766    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:27.767603 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:27.767569    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:29.371000 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:29.370974    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:29.472035 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:29.471990    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:33.632734 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:33.632701    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:44.372447 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:44.372395    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:44.469266 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:44.469225    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:52:44.569890 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:52:44.569836    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:01.703298 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:01.703266    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:01.759018 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:01.758987    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:26.148017 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:26.147981    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:47.165499 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:47.165399    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:47.234261 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:47.234227    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:53.833547 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:53.833503    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:55.984647 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:55.984612    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:56.085043 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:56.084983    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:53:56.187033 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:53:56.186977    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:18.239020 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:18.238976    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:18.345493 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:18.345414    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:19.433135 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:19.433100    2587 kubelet.go:1648] "Image garbage collection succeeded"
Sep 17 20:54:23.908701 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:23.908663    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:24.206073 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:24.206037    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:24.424930 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:24.424901    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:24.956234 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:24.956199    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.163067 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.163015    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.172418 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.172389    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6zcgg"]
Sep 17 20:54:25.175469 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.175452    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.178166 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.178139    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Sep 17 20:54:25.178301 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.178139    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Sep 17 20:54:25.178301 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.178231    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Sep 17 20:54:25.179224 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.179209    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Sep 17 20:54:25.179280 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.179217    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-zkvfx\""
Sep 17 20:54:25.184782 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.184760    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6zcgg"]
Sep 17 20:54:25.232041 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.231960    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-certificates\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.232041 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.232009    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4cpc\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-kube-api-access-q4cpc\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.262836 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.262812    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.332318 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.332282    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-certificates\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.332512 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.332325    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4cpc\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-kube-api-access-q4cpc\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.334816 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.334794    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-certificates\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.341236 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.341210    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4cpc\" (UniqueName: \"kubernetes.io/projected/4d26f726-fb06-4724-aa48-be77925244c0-kube-api-access-q4cpc\") pod \"keda-admission-6c4d864b86-6zcgg\" (UID: \"4d26f726-fb06-4724-aa48-be77925244c0\") " pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.362180 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.362148    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.462039 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.462009    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.487019 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.486942    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:25.576472 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.576445    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.735400 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.735364    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.759337 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.759263    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-6zcgg"]
Sep 17 20:54:25.762596 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:54:25.762570    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d26f726_fb06_4724_aa48_be77925244c0.slice/crio-90397fb8b4ba79f1b687ec5e94d5b8db83403b1172fdb2dddf03131da5268f01 WatchSource:0}: Error finding container 90397fb8b4ba79f1b687ec5e94d5b8db83403b1172fdb2dddf03131da5268f01: Status 404 returned error can't find the container with id 90397fb8b4ba79f1b687ec5e94d5b8db83403b1172fdb2dddf03131da5268f01
Sep 17 20:54:25.764180 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.764165    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 20:54:25.934559 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.934528    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:25.988300 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:25.988269    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:54:26.713122 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:26.713047    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg" event={"ID":"4d26f726-fb06-4724-aa48-be77925244c0","Type":"ContainerStarted","Data":"90397fb8b4ba79f1b687ec5e94d5b8db83403b1172fdb2dddf03131da5268f01"}
Sep 17 20:54:28.721783 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:28.721747    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg" event={"ID":"4d26f726-fb06-4724-aa48-be77925244c0","Type":"ContainerStarted","Data":"b219b442a0b1604413a1e82699bc3af69d50debb503d4b5cb89e61520c487f7b"}
Sep 17 20:54:28.722165 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:28.721866    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:54:28.739849 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:28.739799    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg" podStartSLOduration=1.204136515 podStartE2EDuration="3.739783881s" totalImagesPullingTime="2.535647366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:54:28.738649106 +0000 UTC m=+309.635037200" watchObservedRunningTime="2026-09-17 20:54:28.739783881 +0000 UTC m=+309.636171974"
Sep 17 20:54:49.727682 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:54:49.727650    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-6zcgg"
Sep 17 20:55:16.755231 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:16.755153    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:18.460381 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:18.460345    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:33.836589 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:33.836552    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:33.939701 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:33.939664    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.300345 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.300311    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.386691 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.386658    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"]
Sep 17 20:55:44.388811 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.388789    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.391279 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.391259    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-kq6md\""
Sep 17 20:55:44.391965 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.391937    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Sep 17 20:55:44.392396 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.392377    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Sep 17 20:55:44.392509 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.392491    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Sep 17 20:55:44.397049 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.397029    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.403069 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.403050    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"]
Sep 17 20:55:44.508545 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.508516    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.516617 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.516589    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.516732 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.516654    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdqw\" (UniqueName: \"kubernetes.io/projected/6659ae6b-d988-47cf-8939-bc838da02e53-kube-api-access-nhdqw\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.617100 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.617025    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdqw\" (UniqueName: \"kubernetes.io/projected/6659ae6b-d988-47cf-8939-bc838da02e53-kube-api-access-nhdqw\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.617100 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.617083    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.617265 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:55:44.617167    2587 secret.go:189] Couldn't get secret kserve/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 17 20:55:44.617265 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:55:44.617229    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert podName:6659ae6b-d988-47cf-8939-bc838da02e53 nodeName:}" failed. No retries permitted until 2026-09-17 20:55:45.117213731 +0000 UTC m=+386.013601803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert") pod "llmisvc-controller-manager-645f9c8fd6-sh2f2" (UID: "6659ae6b-d988-47cf-8939-bc838da02e53") : secret "llmisvc-webhook-server-cert" not found
Sep 17 20:55:44.628649 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.628630    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdqw\" (UniqueName: \"kubernetes.io/projected/6659ae6b-d988-47cf-8939-bc838da02e53-kube-api-access-nhdqw\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:44.770150 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.770116    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.873261 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.873177    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:44.997048 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:44.997000    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:45.094088 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.094052    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:45.121229 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.121205    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:45.123589 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.123525    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6659ae6b-d988-47cf-8939-bc838da02e53-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-sh2f2\" (UID: \"6659ae6b-d988-47cf-8939-bc838da02e53\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:45.300626 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.300586    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:45.571213 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.571181    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"]
Sep 17 20:55:45.574302 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:55:45.574278    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6659ae6b_d988_47cf_8939_bc838da02e53.slice/crio-64f4aa941cfcc69db3be84af70962e85c713f17f33f940c759842fae4120f1b3 WatchSource:0}: Error finding container 64f4aa941cfcc69db3be84af70962e85c713f17f33f940c759842fae4120f1b3: Status 404 returned error can't find the container with id 64f4aa941cfcc69db3be84af70962e85c713f17f33f940c759842fae4120f1b3
Sep 17 20:55:45.986927 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:45.986886    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2" event={"ID":"6659ae6b-d988-47cf-8939-bc838da02e53","Type":"ContainerStarted","Data":"64f4aa941cfcc69db3be84af70962e85c713f17f33f940c759842fae4120f1b3"}
Sep 17 20:55:47.996260 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:47.996230    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2" event={"ID":"6659ae6b-d988-47cf-8939-bc838da02e53","Type":"ContainerStarted","Data":"e5b00cbd1a48778d41c567238fb81338b4b47734f0cd6060fd6c88e58899ddbb"}
Sep 17 20:55:47.996653 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:47.996354    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:55:48.013482 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:48.013413    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2" podStartSLOduration=1.776160145 podStartE2EDuration="4.01339692s" totalImagesPullingTime="2.237236775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:55:48.013139081 +0000 UTC m=+388.909527176" watchObservedRunningTime="2026-09-17 20:55:48.01339692 +0000 UTC m=+388.909785013"
Sep 17 20:55:57.535687 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:57.535654    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:55:57.636998 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:55:57.636895    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:08.226150 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:56:08.226118    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:08.332016 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:56:08.331980    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:08.446364 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:56:08.446329    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:08.549550 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:56:08.549477    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:56:19.001908 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:56:19.001874    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-sh2f2"
Sep 17 20:57:01.431535 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.431504    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-g6768"]
Sep 17 20:57:01.434494 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.434478    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.437500 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.437476    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Sep 17 20:57:01.437630 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.437528    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Sep 17 20:57:01.437630 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.437528    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-q6skd\""
Sep 17 20:57:01.448200 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.448178    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-g6768"]
Sep 17 20:57:01.452013 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.451991    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:01.551119 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.551086    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:01.595833 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.595788    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.596018 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.595852    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnd9t\" (UniqueName: \"kubernetes.io/projected/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-kube-api-access-dnd9t\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.596018 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.595906    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-cert\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.650920 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.650889    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:01.697093 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.697019    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-cert\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.697093 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.697074    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.697262 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.697114    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnd9t\" (UniqueName: \"kubernetes.io/projected/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-kube-api-access-dnd9t\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.697262 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:57:01.697226    2587 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found
Sep 17 20:57:01.697337 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:57:01.697290    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs podName:2e5e9082-1c9f-4abd-9847-4ff84e1e8e82 nodeName:}" failed. No retries permitted until 2026-09-17 20:57:02.19727448 +0000 UTC m=+463.093662553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs") pod "odh-model-controller-7b966d7984-g6768" (UID: "2e5e9082-1c9f-4abd-9847-4ff84e1e8e82") : secret "odh-model-controller-metrics-tls" not found
Sep 17 20:57:01.699592 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.699574    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-cert\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.705890 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.705870    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnd9t\" (UniqueName: \"kubernetes.io/projected/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-kube-api-access-dnd9t\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:01.753060 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.753032    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:01.853400 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:01.853362    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:02.201119 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.201075    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:02.203581 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.203546    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e5e9082-1c9f-4abd-9847-4ff84e1e8e82-metrics-certs\") pod \"odh-model-controller-7b966d7984-g6768\" (UID: \"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82\") " pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:02.344586 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.344549    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:02.507530 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.507499    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:02.568403 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.568374    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:57:02.615398 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:02.615373    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-g6768"]
Sep 17 20:57:02.617927 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:57:02.617896    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5e9082_1c9f_4abd_9847_4ff84e1e8e82.slice/crio-8133e67f6cad8e87f2f07eca7e3c53b20852aeba38a481d6046b6d888db85761 WatchSource:0}: Error finding container 8133e67f6cad8e87f2f07eca7e3c53b20852aeba38a481d6046b6d888db85761: Status 404 returned error can't find the container with id 8133e67f6cad8e87f2f07eca7e3c53b20852aeba38a481d6046b6d888db85761
Sep 17 20:57:03.259167 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:03.259132    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-g6768" event={"ID":"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82","Type":"ContainerStarted","Data":"8133e67f6cad8e87f2f07eca7e3c53b20852aeba38a481d6046b6d888db85761"}
Sep 17 20:57:06.271897 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:06.271855    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-g6768" event={"ID":"2e5e9082-1c9f-4abd-9847-4ff84e1e8e82","Type":"ContainerStarted","Data":"6a5cb5fa4190dcee284677a971b591548ac2775dd01a8b62fab87e286ad25232"}
Sep 17 20:57:06.272268 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:06.271954    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:57:06.298767 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:06.298718    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-g6768" podStartSLOduration=2.577427072 podStartE2EDuration="5.298702958s" totalImagesPullingTime="2.721275886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:57:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:57:06.297220544 +0000 UTC m=+467.193608639" watchObservedRunningTime="2026-09-17 20:57:06.298702958 +0000 UTC m=+467.195091052"
Sep 17 20:57:17.277679 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:57:17.277645    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-g6768"
Sep 17 20:58:35.119563 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:35.119522    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:35.888340 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:35.888296    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:35.990813 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:35.990756    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.247650 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.247617    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.481291 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.481248    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.635241 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.635167    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.644434 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.644401    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 20:58:48.647102 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.647082    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.653587 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.653339    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a625d-kube-rbac-proxy-sar-config\""
Sep 17 20:58:48.653740 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.653718    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a625d-predictor-serving-cert\""
Sep 17 20:58:48.654219 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.654192    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tx4r4\""
Sep 17 20:58:48.654302 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.654235    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Sep 17 20:58:48.654362 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.654237    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Sep 17 20:58:48.660515 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.660494    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 20:58:48.696316 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.696280    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w9gt\" (UniqueName: \"kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.696524 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.696321    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.696524 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.696492    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.745898 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.745861    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.797066 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.797022    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.797066 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.797068    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w9gt\" (UniqueName: \"kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.797318 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.797091    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.797318 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:58:48.797242    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a625d-predictor-serving-cert: secret "error-404-isvc-a625d-predictor-serving-cert" not found
Sep 17 20:58:48.797318 ip-10-0-138-134 kubenswrapper[2587]: E0917 20:58:48.797306    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls podName:bf39c692-db71-41a8-886f-38605b70a1c4 nodeName:}" failed. No retries permitted until 2026-09-17 20:58:49.297291963 +0000 UTC m=+570.193680040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls") pod "error-404-isvc-a625d-predictor-7679674998-2mt57" (UID: "bf39c692-db71-41a8-886f-38605b70a1c4") : secret "error-404-isvc-a625d-predictor-serving-cert" not found
Sep 17 20:58:48.797842 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.797817    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.815482 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.815449    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w9gt\" (UniqueName: \"kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:48.850836 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.850801    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:48.960020 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:48.959993    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:49.118786 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.118754    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:49.237162 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.237072    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 20:58:49.300418 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.300384    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:49.303169 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.303147    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") pod \"error-404-isvc-a625d-predictor-7679674998-2mt57\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") " pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:49.557141 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.557051    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:58:49.841938 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:49.841913    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 20:58:49.844523 ip-10-0-138-134 kubenswrapper[2587]: W0917 20:58:49.844494    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf39c692_db71_41a8_886f_38605b70a1c4.slice/crio-09acd4b0e4be8e91dbf65ddffb781b1e17e751e65f011c44e40932abd6fa884c WatchSource:0}: Error finding container 09acd4b0e4be8e91dbf65ddffb781b1e17e751e65f011c44e40932abd6fa884c: Status 404 returned error can't find the container with id 09acd4b0e4be8e91dbf65ddffb781b1e17e751e65f011c44e40932abd6fa884c
Sep 17 20:58:50.646655 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:58:50.646394    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerStarted","Data":"09acd4b0e4be8e91dbf65ddffb781b1e17e751e65f011c44e40932abd6fa884c"}
Sep 17 20:59:02.705711 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:02.705667    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerStarted","Data":"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"}
Sep 17 20:59:04.717713 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:04.717670    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerStarted","Data":"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"}
Sep 17 20:59:04.718184 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:04.717890    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:59:04.718184 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:04.717943    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:59:04.719459 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:04.719408    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:05.721997 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:05.721956    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:10.726110 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:10.726077    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 20:59:10.726742 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:10.726711    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:10.743299 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:10.743235    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podStartSLOduration=8.369270876 podStartE2EDuration="22.74321779s" totalImagesPullingTime="14.373946914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 20:58:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 20:59:04.756333135 +0000 UTC m=+585.652721230" watchObservedRunningTime="2026-09-17 20:59:10.74321779 +0000 UTC m=+591.639605887"
Sep 17 20:59:20.726989 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:20.726948    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:30.726634 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:30.726586    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:40.726954 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:40.726915    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 20:59:50.727108 ip-10-0-138-134 kubenswrapper[2587]: I0917 20:59:50.727019    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.19:8080: connect: connection refused"
Sep 17 21:00:00.727328 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:00.727296    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 21:00:08.151309 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:08.151272    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:09.298287 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:09.298255    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:23.180716 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.180681    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:23.325845 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.325814    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 21:00:23.326060 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.326036    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container" containerID="cri-o://5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3" gracePeriod=30
Sep 17 21:00:23.326150 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.326089    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kube-rbac-proxy" containerID="cri-o://96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66" gracePeriod=30
Sep 17 21:00:23.510012 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.509978    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:00:23.512140 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.512122    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.514930 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.514906    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-23363-predictor-serving-cert\""
Sep 17 21:00:23.515055 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.514947    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-23363-kube-rbac-proxy-sar-config\""
Sep 17 21:00:23.523026 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.523001    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:00:23.643666 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.643631    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42jst\" (UniqueName: \"kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.643666 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.643666    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.643873 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.643760    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.744168 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.744133    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42jst\" (UniqueName: \"kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.744168 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.744172    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.744366 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.744225    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.744366 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:00:23.744341    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-23363-predictor-serving-cert: secret "error-404-isvc-23363-predictor-serving-cert" not found
Sep 17 21:00:23.744475 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:00:23.744409    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls podName:597fa101-f665-46ff-b656-a2f0dec3af5c nodeName:}" failed. No retries permitted until 2026-09-17 21:00:24.244390441 +0000 UTC m=+665.140778526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls") pod "error-404-isvc-23363-predictor-bbcb4956-hk5qt" (UID: "597fa101-f665-46ff-b656-a2f0dec3af5c") : secret "error-404-isvc-23363-predictor-serving-cert" not found
Sep 17 21:00:23.744927 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.744907    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:23.753616 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:23.753586    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42jst\" (UniqueName: \"kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:24.010599 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.010512    2587 generic.go:356] "Generic (PLEG): container finished" podID="bf39c692-db71-41a8-886f-38605b70a1c4" containerID="96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66" exitCode=2
Sep 17 21:00:24.010599 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.010580    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerDied","Data":"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"}
Sep 17 21:00:24.248024 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.247984    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:24.250334 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.250305    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") pod \"error-404-isvc-23363-predictor-bbcb4956-hk5qt\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") " pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:24.423111 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.423016    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:24.498296 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.498268    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:24.697387 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:24.697363    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:00:24.699794 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:00:24.699765    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod597fa101_f665_46ff_b656_a2f0dec3af5c.slice/crio-0839afe039f767c9e42a5b335cdc58577b7677f3fdaf11348813e5240e921418 WatchSource:0}: Error finding container 0839afe039f767c9e42a5b335cdc58577b7677f3fdaf11348813e5240e921418: Status 404 returned error can't find the container with id 0839afe039f767c9e42a5b335cdc58577b7677f3fdaf11348813e5240e921418
Sep 17 21:00:25.018308 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.018266    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerStarted","Data":"ea14c17193d1f9571bf20b35cf2399360f144f9fc0d0ebc87e04465267193342"}
Sep 17 21:00:25.018530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.018349    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:25.018530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.018382    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:25.018530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.018409    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerStarted","Data":"8cb3d62a8176e48c2e85fa3c5e4cd21bc0c1c59ffd580ceeac89e123bb0d3c37"}
Sep 17 21:00:25.018530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.018456    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerStarted","Data":"0839afe039f767c9e42a5b335cdc58577b7677f3fdaf11348813e5240e921418"}
Sep 17 21:00:25.019632 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.019603    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:00:25.036821 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.036753    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podStartSLOduration=2.036737942 podStartE2EDuration="2.036737942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:00:25.036115986 +0000 UTC m=+665.932504081" watchObservedRunningTime="2026-09-17 21:00:25.036737942 +0000 UTC m=+665.933126036"
Sep 17 21:00:25.722673 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:25.722622    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.19:8643/healthz\": dial tcp 10.133.0.19:8643: connect: connection refused"
Sep 17 21:00:26.022708 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.022613    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:00:26.866600 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.866574    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 21:00:26.968081 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.967997    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config\") pod \"bf39c692-db71-41a8-886f-38605b70a1c4\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") "
Sep 17 21:00:26.968081 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.968038    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") pod \"bf39c692-db71-41a8-886f-38605b70a1c4\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") "
Sep 17 21:00:26.968081 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.968082    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w9gt\" (UniqueName: \"kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt\") pod \"bf39c692-db71-41a8-886f-38605b70a1c4\" (UID: \"bf39c692-db71-41a8-886f-38605b70a1c4\") "
Sep 17 21:00:26.968444 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.968403    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bf39c692-db71-41a8-886f-38605b70a1c4" (UID: "bf39c692-db71-41a8-886f-38605b70a1c4"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:00:26.970318 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.970292    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bf39c692-db71-41a8-886f-38605b70a1c4" (UID: "bf39c692-db71-41a8-886f-38605b70a1c4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:00:26.970318 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:26.970287    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt" (OuterVolumeSpecName: "kube-api-access-6w9gt") pod "bf39c692-db71-41a8-886f-38605b70a1c4" (UID: "bf39c692-db71-41a8-886f-38605b70a1c4"). InnerVolumeSpecName "kube-api-access-6w9gt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:00:27.027396 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.027361    2587 generic.go:356] "Generic (PLEG): container finished" podID="bf39c692-db71-41a8-886f-38605b70a1c4" containerID="5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3" exitCode=0
Sep 17 21:00:27.027611 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.027405    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerDied","Data":"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"}
Sep 17 21:00:27.027611 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.027468    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57" event={"ID":"bf39c692-db71-41a8-886f-38605b70a1c4","Type":"ContainerDied","Data":"09acd4b0e4be8e91dbf65ddffb781b1e17e751e65f011c44e40932abd6fa884c"}
Sep 17 21:00:27.027611 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.027498    2587 scope.go:117] "RemoveContainer" containerID="96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"
Sep 17 21:00:27.027611 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.027409    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"
Sep 17 21:00:27.035535 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.035515    2587 scope.go:117] "RemoveContainer" containerID="5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"
Sep 17 21:00:27.042852 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.042833    2587 scope.go:117] "RemoveContainer" containerID="96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"
Sep 17 21:00:27.043104 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:00:27.043084    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66\": container with ID starting with 96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66 not found: ID does not exist" containerID="96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"
Sep 17 21:00:27.043153 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.043113    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66"} err="failed to get container status \"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66\": rpc error: code = NotFound desc = could not find container \"96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66\": container with ID starting with 96d5df5fe5769964124675be4ddd7a927520d1d60a342af42d6057a7cf3a6e66 not found: ID does not exist"
Sep 17 21:00:27.043153 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.043142    2587 scope.go:117] "RemoveContainer" containerID="5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"
Sep 17 21:00:27.043385 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:00:27.043367    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3\": container with ID starting with 5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3 not found: ID does not exist" containerID="5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"
Sep 17 21:00:27.043446 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.043392    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3"} err="failed to get container status \"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3\": rpc error: code = NotFound desc = could not find container \"5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3\": container with ID starting with 5178cccd6b968e17e0c851d8e5ee617427a54bec6e7fd81e89d28188556746f3 not found: ID does not exist"
Sep 17 21:00:27.048847 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.048826    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 21:00:27.053236 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.053214    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a625d-predictor-7679674998-2mt57"]
Sep 17 21:00:27.068808 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.068786    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bf39c692-db71-41a8-886f-38605b70a1c4-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:00:27.068808 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.068807    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bf39c692-db71-41a8-886f-38605b70a1c4-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:00:27.068954 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.068819    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w9gt\" (UniqueName: \"kubernetes.io/projected/bf39c692-db71-41a8-886f-38605b70a1c4-kube-api-access-6w9gt\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:00:27.561494 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:27.561467    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" path="/var/lib/kubelet/pods/bf39c692-db71-41a8-886f-38605b70a1c4/volumes"
Sep 17 21:00:31.027759 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:31.027720    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:00:31.028342 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:31.028309    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:00:32.786872 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:32.786834    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:41.028597 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:41.028544    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:00:48.336115 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:48.336083    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:48.404448 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:48.404399    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:00:51.028542 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:00:51.028492    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:01:01.028326 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:01.028283    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:01:08.262493 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:08.262456    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:08.430488 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:08.430449    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:11.028476 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:11.028417    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:01:19.784950 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:19.784872    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:19.877163 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:19.877129    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:19.943154 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:19.943122    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:20.007603 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.007574    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:20.101855 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.101778    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:01:20.102127 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102114    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kube-rbac-proxy"
Sep 17 21:01:20.102167 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102130    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kube-rbac-proxy"
Sep 17 21:01:20.102167 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102141    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container"
Sep 17 21:01:20.102167 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102147    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container"
Sep 17 21:01:20.102254 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102194    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kube-rbac-proxy"
Sep 17 21:01:20.102254 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.102205    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf39c692-db71-41a8-886f-38605b70a1c4" containerName="kserve-container"
Sep 17 21:01:20.106300 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.106282    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.107581 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.107556    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:20.109061 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.109039    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ed647-predictor-serving-cert\""
Sep 17 21:01:20.109164 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.109077    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-ed647-kube-rbac-proxy-sar-config\""
Sep 17 21:01:20.115204 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.115181    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:01:20.166750 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.166707    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.166928 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.166829    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pznn6\" (UniqueName: \"kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.166928 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.166911    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.267461 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.267398    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pznn6\" (UniqueName: \"kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.267632 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.267472    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.267632 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.267498    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.267632 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:01:20.267583    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-ed647-predictor-serving-cert: secret "error-404-isvc-ed647-predictor-serving-cert" not found
Sep 17 21:01:20.267747 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:01:20.267654    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls podName:87321e3b-b69f-452f-99fb-721db9638941 nodeName:}" failed. No retries permitted until 2026-09-17 21:01:20.767633291 +0000 UTC m=+721.664021377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls") pod "error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" (UID: "87321e3b-b69f-452f-99fb-721db9638941") : secret "error-404-isvc-ed647-predictor-serving-cert" not found
Sep 17 21:01:20.268146 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.268123    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.278443 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.278390    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pznn6\" (UniqueName: \"kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.312909 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.312874    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:20.553385 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.553346    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:20.769765 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.769716    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:20.772213 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:20.772190    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") pod \"error-404-isvc-ed647-predictor-7fbb665cd-xwzbp\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") " pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:21.016517 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:21.016474    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:21.029992 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:21.029967    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:01:21.290309 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:21.290236    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:01:21.293774 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:01:21.293746    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87321e3b_b69f_452f_99fb_721db9638941.slice/crio-e1c72b4df99b476b24189eaa3d255e6298b42ffb86c0bc50a0ba1bd04afbd2cf WatchSource:0}: Error finding container e1c72b4df99b476b24189eaa3d255e6298b42ffb86c0bc50a0ba1bd04afbd2cf: Status 404 returned error can't find the container with id e1c72b4df99b476b24189eaa3d255e6298b42ffb86c0bc50a0ba1bd04afbd2cf
Sep 17 21:01:22.230873 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:22.230839    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerStarted","Data":"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"}
Sep 17 21:01:22.231249 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:22.230906    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerStarted","Data":"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"}
Sep 17 21:01:22.231249 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:22.230937    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerStarted","Data":"e1c72b4df99b476b24189eaa3d255e6298b42ffb86c0bc50a0ba1bd04afbd2cf"}
Sep 17 21:01:22.231249 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:22.230973    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:22.265227 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:22.265174    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podStartSLOduration=2.265159029 podStartE2EDuration="2.265159029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:01:22.262467893 +0000 UTC m=+723.158855986" watchObservedRunningTime="2026-09-17 21:01:22.265159029 +0000 UTC m=+723.161547124"
Sep 17 21:01:23.235000 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:23.234971    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:23.236251 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:23.236226    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:01:24.238970 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:24.238931    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:01:29.250307 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:29.250274    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:01:29.250940 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:29.250908    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:01:33.404838 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:33.404808    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:33.531574 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:33.531542    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:01:39.250797 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:39.250754    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:01:49.251088 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:49.251044    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:01:59.251584 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:01:59.251546    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:02:02.322246 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:02.322209    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:02:09.251671 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:09.251629    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused"
Sep 17 21:02:19.252166 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:19.252132    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:02:29.198844 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:29.198806    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:02:29.317610 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:29.317575    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:02:29.984070 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:29.984027    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:02:30.086292 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:02:30.086261    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:23.166762 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:06:23.166729    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:06:48.114974 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:06:48.114893    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.515153 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.515111    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.617312 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.617279    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.693738 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.693700    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:09:48.693944 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.693920    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" containerID="cri-o://8cb3d62a8176e48c2e85fa3c5e4cd21bc0c1c59ffd580ceeac89e123bb0d3c37" gracePeriod=30
Sep 17 21:09:48.694026 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.693994    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kube-rbac-proxy" containerID="cri-o://ea14c17193d1f9571bf20b35cf2399360f144f9fc0d0ebc87e04465267193342" gracePeriod=30
Sep 17 21:09:48.697937 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.697887    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.751161 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.751126    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:09:48.754329 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.754313    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.756886 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.756867    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-46851-predictor-serving-cert\""
Sep 17 21:09:48.757087 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.757073    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-46851-kube-rbac-proxy-sar-config\""
Sep 17 21:09:48.763493 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.763471    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:09:48.766675 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.766624    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.813677 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.813654    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5prt8\" (UniqueName: \"kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.813756 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.813691    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.813756 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.813714    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.866735 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.866698    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:48.915091 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.915055    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5prt8\" (UniqueName: \"kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.915222 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.915102    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.915222 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.915123    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.915314 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:09:48.915234    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-46851-predictor-serving-cert: secret "error-404-isvc-46851-predictor-serving-cert" not found
Sep 17 21:09:48.915356 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:09:48.915318    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls podName:31a88c45-0456-464e-a1b8-7af05168da5f nodeName:}" failed. No retries permitted until 2026-09-17 21:09:49.415297889 +0000 UTC m=+1230.311685961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls") pod "error-404-isvc-46851-predictor-7f76877c8-jkqxj" (UID: "31a88c45-0456-464e-a1b8-7af05168da5f") : secret "error-404-isvc-46851-predictor-serving-cert" not found
Sep 17 21:09:48.915791 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.915773    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:48.926134 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:48.926112    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5prt8\" (UniqueName: \"kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:49.107410 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.107333    2587 generic.go:356] "Generic (PLEG): container finished" podID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerID="ea14c17193d1f9571bf20b35cf2399360f144f9fc0d0ebc87e04465267193342" exitCode=2
Sep 17 21:09:49.107554 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.107404    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerDied","Data":"ea14c17193d1f9571bf20b35cf2399360f144f9fc0d0ebc87e04465267193342"}
Sep 17 21:09:49.243075 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.243030    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:49.419823 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.419730    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:49.422235 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.422198    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") pod \"error-404-isvc-46851-predictor-7f76877c8-jkqxj\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") " pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:49.664604 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.664552    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:49.928224 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.928140    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:09:49.939842 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:49.939661    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:09:49.942395 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:09:49.942361    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31a88c45_0456_464e_a1b8_7af05168da5f.slice/crio-4778689c39931ee903b37001738f3218cfe52ce7e9d7fffa80457c090395040d WatchSource:0}: Error finding container 4778689c39931ee903b37001738f3218cfe52ce7e9d7fffa80457c090395040d: Status 404 returned error can't find the container with id 4778689c39931ee903b37001738f3218cfe52ce7e9d7fffa80457c090395040d
Sep 17 21:09:50.113128 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.113089    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerStarted","Data":"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"}
Sep 17 21:09:50.113309 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.113166    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerStarted","Data":"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"}
Sep 17 21:09:50.113309 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.113204    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerStarted","Data":"4778689c39931ee903b37001738f3218cfe52ce7e9d7fffa80457c090395040d"}
Sep 17 21:09:50.113448 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.113390    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:50.113514 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.113465    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:50.115068 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.115036    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:09:50.132156 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:50.132108    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podStartSLOduration=2.132092626 podStartE2EDuration="2.132092626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:09:50.130349232 +0000 UTC m=+1231.026737353" watchObservedRunningTime="2026-09-17 21:09:50.132092626 +0000 UTC m=+1231.028480719"
Sep 17 21:09:51.023540 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:51.023500    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.20:8643/healthz\": dial tcp 10.133.0.20:8643: connect: connection refused"
Sep 17 21:09:51.028981 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:51.028954    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.20:8080: connect: connection refused"
Sep 17 21:09:51.118166 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:51.118115    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:09:52.124439 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.124395    2587 generic.go:356] "Generic (PLEG): container finished" podID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerID="8cb3d62a8176e48c2e85fa3c5e4cd21bc0c1c59ffd580ceeac89e123bb0d3c37" exitCode=0
Sep 17 21:09:52.124769 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.124459    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerDied","Data":"8cb3d62a8176e48c2e85fa3c5e4cd21bc0c1c59ffd580ceeac89e123bb0d3c37"}
Sep 17 21:09:52.249786 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.249757    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:09:52.343626 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.343543    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-42jst\" (UniqueName: \"kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst\") pod \"597fa101-f665-46ff-b656-a2f0dec3af5c\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") "
Sep 17 21:09:52.343626 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.343587    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config\") pod \"597fa101-f665-46ff-b656-a2f0dec3af5c\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") "
Sep 17 21:09:52.343626 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.343614    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") pod \"597fa101-f665-46ff-b656-a2f0dec3af5c\" (UID: \"597fa101-f665-46ff-b656-a2f0dec3af5c\") "
Sep 17 21:09:52.343928 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.343906    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "597fa101-f665-46ff-b656-a2f0dec3af5c" (UID: "597fa101-f665-46ff-b656-a2f0dec3af5c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:09:52.345742 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.345718    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "597fa101-f665-46ff-b656-a2f0dec3af5c" (UID: "597fa101-f665-46ff-b656-a2f0dec3af5c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:09:52.345833 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.345811    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst" (OuterVolumeSpecName: "kube-api-access-42jst") pod "597fa101-f665-46ff-b656-a2f0dec3af5c" (UID: "597fa101-f665-46ff-b656-a2f0dec3af5c"). InnerVolumeSpecName "kube-api-access-42jst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:09:52.444602 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.444568    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42jst\" (UniqueName: \"kubernetes.io/projected/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-api-access-42jst\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:09:52.444602 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.444596    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/597fa101-f665-46ff-b656-a2f0dec3af5c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:09:52.444602 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:52.444606    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/597fa101-f665-46ff-b656-a2f0dec3af5c-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:09:53.130208 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.130181    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"
Sep 17 21:09:53.130208 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.130188    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt" event={"ID":"597fa101-f665-46ff-b656-a2f0dec3af5c","Type":"ContainerDied","Data":"0839afe039f767c9e42a5b335cdc58577b7677f3fdaf11348813e5240e921418"}
Sep 17 21:09:53.130689 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.130246    2587 scope.go:117] "RemoveContainer" containerID="ea14c17193d1f9571bf20b35cf2399360f144f9fc0d0ebc87e04465267193342"
Sep 17 21:09:53.139775 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.139757    2587 scope.go:117] "RemoveContainer" containerID="8cb3d62a8176e48c2e85fa3c5e4cd21bc0c1c59ffd580ceeac89e123bb0d3c37"
Sep 17 21:09:53.157869 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.157844    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:09:53.159869 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.159848    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-23363-predictor-bbcb4956-hk5qt"]
Sep 17 21:09:53.561512 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:53.561478    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" path="/var/lib/kubelet/pods/597fa101-f665-46ff-b656-a2f0dec3af5c/volumes"
Sep 17 21:09:56.122938 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:56.122909    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:09:56.123525 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:09:56.123496    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:10:06.124345 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:06.124298    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:10:16.123870 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:16.123779    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:10:26.124211 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:26.124170    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:10:36.124209 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:36.124166    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.22:8080: connect: connection refused"
Sep 17 21:10:45.563513 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.563476    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:45.656499 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.656459    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:45.664886 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.664861    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:10:45.665135 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.665104    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container" containerID="cri-o://416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b" gracePeriod=30
Sep 17 21:10:45.665281 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.665129    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kube-rbac-proxy" containerID="cri-o://c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6" gracePeriod=30
Sep 17 21:10:45.779518 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.779485    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:45.849665 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.849566    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:45.855495 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855469    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:10:45.855777 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855765    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kube-rbac-proxy"
Sep 17 21:10:45.855822 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855778    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kube-rbac-proxy"
Sep 17 21:10:45.855822 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855796    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container"
Sep 17 21:10:45.855822 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855802    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container"
Sep 17 21:10:45.855926 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855850    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kube-rbac-proxy"
Sep 17 21:10:45.855926 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.855860    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="597fa101-f665-46ff-b656-a2f0dec3af5c" containerName="kserve-container"
Sep 17 21:10:45.858881 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.858864    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.861413 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.861388    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-62e6f-kube-rbac-proxy-sar-config\""
Sep 17 21:10:45.861538 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.861457    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-62e6f-predictor-serving-cert\""
Sep 17 21:10:45.869032 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.869008    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:10:45.884986 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.884952    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g46m\" (UniqueName: \"kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.885131 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.885009    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.885131 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.885043    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.985629 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.985599    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.985805 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.985640    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.985805 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.985723    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g46m\" (UniqueName: \"kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.986372 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.986344    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.988032 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.988015    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.994045 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.994024    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g46m\" (UniqueName: \"kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m\") pod \"error-404-isvc-62e6f-predictor-6984c9779b-6qqq9\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") " pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:45.995268 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:45.995242    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:46.098199 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.098164    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:46.124621 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.124551    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:10:46.169381 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.169338    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:46.331777 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.331743    2587 generic.go:356] "Generic (PLEG): container finished" podID="87321e3b-b69f-452f-99fb-721db9638941" containerID="c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6" exitCode=2
Sep 17 21:10:46.331914 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.331798    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerDied","Data":"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"}
Sep 17 21:10:46.439614 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:46.439571    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:10:46.441795 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:10:46.441772    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d635a8_3d5f_4336_a9b3_20472d79a220.slice/crio-7654c75fc8a22d25413aa4a986e1eda0fe5b57e35d8222bb08c02ababe31ef0a WatchSource:0}: Error finding container 7654c75fc8a22d25413aa4a986e1eda0fe5b57e35d8222bb08c02ababe31ef0a: Status 404 returned error can't find the container with id 7654c75fc8a22d25413aa4a986e1eda0fe5b57e35d8222bb08c02ababe31ef0a
Sep 17 21:10:47.337566 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:47.337532    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerStarted","Data":"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"}
Sep 17 21:10:47.337986 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:47.337592    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerStarted","Data":"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"}
Sep 17 21:10:47.337986 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:47.337617    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:47.337986 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:47.337637    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerStarted","Data":"7654c75fc8a22d25413aa4a986e1eda0fe5b57e35d8222bb08c02ababe31ef0a"}
Sep 17 21:10:47.358143 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:47.358099    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podStartSLOduration=2.358084614 podStartE2EDuration="2.358084614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:10:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:10:47.356986737 +0000 UTC m=+1288.253374833" watchObservedRunningTime="2026-09-17 21:10:47.358084614 +0000 UTC m=+1288.254472707"
Sep 17 21:10:48.341980 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:48.341949    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:48.343160 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:48.343130    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:10:49.120003 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.119975    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:10:49.208931 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.208903    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") pod \"87321e3b-b69f-452f-99fb-721db9638941\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") "
Sep 17 21:10:49.209084 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.208951    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pznn6\" (UniqueName: \"kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6\") pod \"87321e3b-b69f-452f-99fb-721db9638941\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") "
Sep 17 21:10:49.209084 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.208999    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config\") pod \"87321e3b-b69f-452f-99fb-721db9638941\" (UID: \"87321e3b-b69f-452f-99fb-721db9638941\") "
Sep 17 21:10:49.209399 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.209375    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "87321e3b-b69f-452f-99fb-721db9638941" (UID: "87321e3b-b69f-452f-99fb-721db9638941"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:10:49.210921 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.210900    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "87321e3b-b69f-452f-99fb-721db9638941" (UID: "87321e3b-b69f-452f-99fb-721db9638941"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:10:49.211018 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.211002    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6" (OuterVolumeSpecName: "kube-api-access-pznn6") pod "87321e3b-b69f-452f-99fb-721db9638941" (UID: "87321e3b-b69f-452f-99fb-721db9638941"). InnerVolumeSpecName "kube-api-access-pznn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:10:49.309481 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.309442    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/87321e3b-b69f-452f-99fb-721db9638941-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:10:49.309481 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.309477    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pznn6\" (UniqueName: \"kubernetes.io/projected/87321e3b-b69f-452f-99fb-721db9638941-kube-api-access-pznn6\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:10:49.309654 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.309491    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/87321e3b-b69f-452f-99fb-721db9638941-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:10:49.352273 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352236    2587 generic.go:356] "Generic (PLEG): container finished" podID="87321e3b-b69f-452f-99fb-721db9638941" containerID="416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b" exitCode=0
Sep 17 21:10:49.352716 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352297    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"
Sep 17 21:10:49.352716 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352321    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerDied","Data":"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"}
Sep 17 21:10:49.352716 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352441    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp" event={"ID":"87321e3b-b69f-452f-99fb-721db9638941","Type":"ContainerDied","Data":"e1c72b4df99b476b24189eaa3d255e6298b42ffb86c0bc50a0ba1bd04afbd2cf"}
Sep 17 21:10:49.352716 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352484    2587 scope.go:117] "RemoveContainer" containerID="c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"
Sep 17 21:10:49.352931 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.352847    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:10:49.362911 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.362892    2587 scope.go:117] "RemoveContainer" containerID="416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"
Sep 17 21:10:49.370463 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.370436    2587 scope.go:117] "RemoveContainer" containerID="c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"
Sep 17 21:10:49.370726 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:10:49.370706    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6\": container with ID starting with c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6 not found: ID does not exist" containerID="c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"
Sep 17 21:10:49.370789 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.370735    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6"} err="failed to get container status \"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6\": rpc error: code = NotFound desc = could not find container \"c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6\": container with ID starting with c05c33c3e82c973400ad0244d0b818c35e0c90cbe6d5c62fa90bffc33ecc97b6 not found: ID does not exist"
Sep 17 21:10:49.370789 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.370751    2587 scope.go:117] "RemoveContainer" containerID="416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"
Sep 17 21:10:49.371021 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:10:49.370993    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b\": container with ID starting with 416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b not found: ID does not exist" containerID="416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"
Sep 17 21:10:49.371105 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.371027    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b"} err="failed to get container status \"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b\": rpc error: code = NotFound desc = could not find container \"416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b\": container with ID starting with 416da1bed7b0d58732ee1b437f5944af8239f24ad9de803e36ce2065bbbf9c5b not found: ID does not exist"
Sep 17 21:10:49.373121 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.373099    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:10:49.376250 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.376227    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-ed647-predictor-7fbb665cd-xwzbp"]
Sep 17 21:10:49.562702 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:49.562624    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87321e3b-b69f-452f-99fb-721db9638941" path="/var/lib/kubelet/pods/87321e3b-b69f-452f-99fb-721db9638941/volumes"
Sep 17 21:10:54.357309 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:54.357279    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:10:54.357810 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:54.357782    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:10:58.779893 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:58.779856    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:10:58.934668 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:10:58.934631    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:04.357781 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:04.357739    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:11:09.397589 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.397553    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:09.508710 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.508675    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:09.516320 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.516291    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:11:09.516561 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.516533    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container" containerID="cri-o://fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea" gracePeriod=30
Sep 17 21:11:09.516647 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.516588    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kube-rbac-proxy" containerID="cri-o://b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd" gracePeriod=30
Sep 17 21:11:09.605358 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.605321    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:09.675586 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.675506    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:09.709882 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.709845    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:11:09.710152 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710140    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container"
Sep 17 21:11:09.710195 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710153    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container"
Sep 17 21:11:09.710195 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710175    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kube-rbac-proxy"
Sep 17 21:11:09.710195 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710181    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kube-rbac-proxy"
Sep 17 21:11:09.710288 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710229    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kube-rbac-proxy"
Sep 17 21:11:09.710288 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.710238    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="87321e3b-b69f-452f-99fb-721db9638941" containerName="kserve-container"
Sep 17 21:11:09.713193 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.713174    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.716116 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.716098    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-31e6a-kube-rbac-proxy-sar-config\""
Sep 17 21:11:09.716116 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.716111    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-31e6a-predictor-serving-cert\""
Sep 17 21:11:09.724356 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.724329    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:11:09.744911 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.744880    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrqj\" (UniqueName: \"kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.745045 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.744944    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.745045 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.745004    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.777347 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.777317    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:09.845946 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.845911    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.846119 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.846067    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrqj\" (UniqueName: \"kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.846119 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.846106    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.846231 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:09.846217    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-serving-cert: secret "error-404-isvc-31e6a-predictor-serving-cert" not found
Sep 17 21:11:09.846292 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:09.846282    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls podName:5b456083-3b26-453e-b265-f7bc0ba97343 nodeName:}" failed. No retries permitted until 2026-09-17 21:11:10.346264098 +0000 UTC m=+1311.242652185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls") pod "error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" (UID: "5b456083-3b26-453e-b265-f7bc0ba97343") : secret "error-404-isvc-31e6a-predictor-serving-cert" not found
Sep 17 21:11:09.846649 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.846631    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:09.857300 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:09.857272    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrqj\" (UniqueName: \"kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:10.081189 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:10.081157    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:10.349505 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:10.349385    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:10.349657 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:10.349526    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-serving-cert: secret "error-404-isvc-31e6a-predictor-serving-cert" not found
Sep 17 21:11:10.349657 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:10.349614    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls podName:5b456083-3b26-453e-b265-f7bc0ba97343 nodeName:}" failed. No retries permitted until 2026-09-17 21:11:11.3495979 +0000 UTC m=+1312.245985971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls") pod "error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" (UID: "5b456083-3b26-453e-b265-f7bc0ba97343") : secret "error-404-isvc-31e6a-predictor-serving-cert" not found
Sep 17 21:11:10.431565 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:10.431531    2587 generic.go:356] "Generic (PLEG): container finished" podID="31a88c45-0456-464e-a1b8-7af05168da5f" containerID="b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd" exitCode=2
Sep 17 21:11:10.431909 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:10.431604    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerDied","Data":"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"}
Sep 17 21:11:10.439014 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:10.438983    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:11.118699 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:11.118648    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.22:8643/healthz\": dial tcp 10.133.0.22:8643: connect: connection refused"
Sep 17 21:11:11.355999 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:11.355966    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:11.358287 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:11.358265    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") " pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:11.525921 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:11.525881    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:11.798960 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:11.798884    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:11:11.802148 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:11:11.802121    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b456083_3b26_453e_b265_f7bc0ba97343.slice/crio-d52114c0dcea754dc823d2743d381e3abfd66e8a10f89c3aa40bb1ca0f42d927 WatchSource:0}: Error finding container d52114c0dcea754dc823d2743d381e3abfd66e8a10f89c3aa40bb1ca0f42d927: Status 404 returned error can't find the container with id d52114c0dcea754dc823d2743d381e3abfd66e8a10f89c3aa40bb1ca0f42d927
Sep 17 21:11:12.440438 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:12.440397    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerStarted","Data":"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"}
Sep 17 21:11:12.440617 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:12.440473    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerStarted","Data":"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"}
Sep 17 21:11:12.440617 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:12.440494    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerStarted","Data":"d52114c0dcea754dc823d2743d381e3abfd66e8a10f89c3aa40bb1ca0f42d927"}
Sep 17 21:11:12.440617 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:12.440568    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:12.458676 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:12.458631    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podStartSLOduration=3.45861629 podStartE2EDuration="3.45861629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:11:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:11:12.457233086 +0000 UTC m=+1313.353621180" watchObservedRunningTime="2026-09-17 21:11:12.45861629 +0000 UTC m=+1313.355004383"
Sep 17 21:11:13.062373 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.062350    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:11:13.066226 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.066205    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config\") pod \"31a88c45-0456-464e-a1b8-7af05168da5f\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") "
Sep 17 21:11:13.066331 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.066293    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") pod \"31a88c45-0456-464e-a1b8-7af05168da5f\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") "
Sep 17 21:11:13.066394 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.066349    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5prt8\" (UniqueName: \"kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8\") pod \"31a88c45-0456-464e-a1b8-7af05168da5f\" (UID: \"31a88c45-0456-464e-a1b8-7af05168da5f\") "
Sep 17 21:11:13.066527 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.066507    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "31a88c45-0456-464e-a1b8-7af05168da5f" (UID: "31a88c45-0456-464e-a1b8-7af05168da5f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:11:13.066586 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.066573    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/31a88c45-0456-464e-a1b8-7af05168da5f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:11:13.068299 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.068271    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8" (OuterVolumeSpecName: "kube-api-access-5prt8") pod "31a88c45-0456-464e-a1b8-7af05168da5f" (UID: "31a88c45-0456-464e-a1b8-7af05168da5f"). InnerVolumeSpecName "kube-api-access-5prt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:11:13.068387 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.068274    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "31a88c45-0456-464e-a1b8-7af05168da5f" (UID: "31a88c45-0456-464e-a1b8-7af05168da5f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:11:13.167728 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.167649    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/31a88c45-0456-464e-a1b8-7af05168da5f-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:11:13.167728 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.167677    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5prt8\" (UniqueName: \"kubernetes.io/projected/31a88c45-0456-464e-a1b8-7af05168da5f-kube-api-access-5prt8\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:11:13.445348 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445266    2587 generic.go:356] "Generic (PLEG): container finished" podID="31a88c45-0456-464e-a1b8-7af05168da5f" containerID="fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea" exitCode=0
Sep 17 21:11:13.445348 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445327    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"
Sep 17 21:11:13.445540 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445359    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerDied","Data":"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"}
Sep 17 21:11:13.445540 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445450    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj" event={"ID":"31a88c45-0456-464e-a1b8-7af05168da5f","Type":"ContainerDied","Data":"4778689c39931ee903b37001738f3218cfe52ce7e9d7fffa80457c090395040d"}
Sep 17 21:11:13.445540 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445491    2587 scope.go:117] "RemoveContainer" containerID="b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"
Sep 17 21:11:13.445865 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.445834    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:13.447304 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.447280    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:13.456302 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.455160    2587 scope.go:117] "RemoveContainer" containerID="fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"
Sep 17 21:11:13.464144 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.464127    2587 scope.go:117] "RemoveContainer" containerID="b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"
Sep 17 21:11:13.464392 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:13.464371    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd\": container with ID starting with b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd not found: ID does not exist" containerID="b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"
Sep 17 21:11:13.464459 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.464400    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd"} err="failed to get container status \"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd\": rpc error: code = NotFound desc = could not find container \"b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd\": container with ID starting with b8f9126b0cf35143bf5a5f647875c1a5533628e01698df3edcdbec89f30825fd not found: ID does not exist"
Sep 17 21:11:13.464459 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.464441    2587 scope.go:117] "RemoveContainer" containerID="fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"
Sep 17 21:11:13.464684 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:11:13.464666    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea\": container with ID starting with fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea not found: ID does not exist" containerID="fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"
Sep 17 21:11:13.464737 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.464690    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea"} err="failed to get container status \"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea\": rpc error: code = NotFound desc = could not find container \"fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea\": container with ID starting with fb1ded35cf1c8b15cdde09b173bed0521bab41fd27e5fd196e8618b0ae451aea not found: ID does not exist"
Sep 17 21:11:13.472124 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.472103    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:11:13.476089 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.476069    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-46851-predictor-7f76877c8-jkqxj"]
Sep 17 21:11:13.561992 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:13.561963    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" path="/var/lib/kubelet/pods/31a88c45-0456-464e-a1b8-7af05168da5f/volumes"
Sep 17 21:11:14.358669 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:14.358624    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:11:14.451552 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:14.451513    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:19.456174 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:19.456147    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:11:19.456607 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:19.456583    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:24.358473 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:24.358414    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:11:29.457011 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:29.456967    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:34.358141 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:34.358097    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.23:8080: connect: connection refused"
Sep 17 21:11:39.457520 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:39.457469    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:44.358666 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:44.358573    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:11:49.457569 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:49.457526    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:11:55.781721 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:55.781674    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:57.521729 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:57.521690    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:11:59.456763 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:11:59.456724    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.24:8080: connect: connection refused"
Sep 17 21:12:06.905554 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:06.905509    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.145264 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.145224    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.174904 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.174831    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:12:07.175107 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.175078    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container" containerID="cri-o://0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b" gracePeriod=30
Sep 17 21:12:07.175166 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.175097    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kube-rbac-proxy" containerID="cri-o://eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51" gracePeriod=30
Sep 17 21:12:07.241215 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.241174    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.348237 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.348205    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.375722 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.375689    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:12:07.376030 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376017    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container"
Sep 17 21:12:07.376082 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376031    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container"
Sep 17 21:12:07.376082 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376041    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kube-rbac-proxy"
Sep 17 21:12:07.376082 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376046    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kube-rbac-proxy"
Sep 17 21:12:07.376176 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376094    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kserve-container"
Sep 17 21:12:07.376176 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.376102    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="31a88c45-0456-464e-a1b8-7af05168da5f" containerName="kube-rbac-proxy"
Sep 17 21:12:07.379025 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.379004    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.382026 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.382008    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b5a48-kube-rbac-proxy-sar-config\""
Sep 17 21:12:07.382361 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.382335    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-b5a48-predictor-serving-cert\""
Sep 17 21:12:07.390050 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.390025    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:12:07.424073 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.424044    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.503766 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.503736    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.503917 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.503778    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svllc\" (UniqueName: \"kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.503917 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.503843    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.508042 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.508023    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:07.604969 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.604935    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.605148 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.604977    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svllc\" (UniqueName: \"kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.605148 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.605023    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.605614 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.605588    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.607405 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.607387    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.613794 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.613769    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svllc\" (UniqueName: \"kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc\") pod \"error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") " pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.645585 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.645554    2587 generic.go:356] "Generic (PLEG): container finished" podID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerID="eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51" exitCode=2
Sep 17 21:12:07.645726 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.645602    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerDied","Data":"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"}
Sep 17 21:12:07.688947 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.688917    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:07.958099 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:07.958070    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:12:07.960566 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:12:07.960538    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod852e4935_ae82_4b04_96c6_2a62961ab4b7.slice/crio-ecabc0781e22a7fcaa63550f6f9b804b0f79e2eaf1c5cb2edc41bfc27a0143ec WatchSource:0}: Error finding container ecabc0781e22a7fcaa63550f6f9b804b0f79e2eaf1c5cb2edc41bfc27a0143ec: Status 404 returned error can't find the container with id ecabc0781e22a7fcaa63550f6f9b804b0f79e2eaf1c5cb2edc41bfc27a0143ec
Sep 17 21:12:08.650918 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:08.650880    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerStarted","Data":"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"}
Sep 17 21:12:08.651067 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:08.650949    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerStarted","Data":"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"}
Sep 17 21:12:08.651067 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:08.650983    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerStarted","Data":"ecabc0781e22a7fcaa63550f6f9b804b0f79e2eaf1c5cb2edc41bfc27a0143ec"}
Sep 17 21:12:08.651067 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:08.651054    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:08.670582 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:08.670522    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podStartSLOduration=1.670506369 podStartE2EDuration="1.670506369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:12:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:12:08.669414101 +0000 UTC m=+1369.565802195" watchObservedRunningTime="2026-09-17 21:12:08.670506369 +0000 UTC m=+1369.566894462"
Sep 17 21:12:09.353920 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:09.353875    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.23:8643/healthz\": dial tcp 10.133.0.23:8643: connect: connection refused"
Sep 17 21:12:09.457553 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:09.457521    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:12:09.654512 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:09.654432    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:09.655666 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:09.655641    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:10.658408 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.658365    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:10.824206 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.824182    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:12:10.929005 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.928935    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config\") pod \"77d635a8-3d5f-4336-a9b3-20472d79a220\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") "
Sep 17 21:12:10.929005 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.928976    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2g46m\" (UniqueName: \"kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m\") pod \"77d635a8-3d5f-4336-a9b3-20472d79a220\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") "
Sep 17 21:12:10.929005 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.928996    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls\") pod \"77d635a8-3d5f-4336-a9b3-20472d79a220\" (UID: \"77d635a8-3d5f-4336-a9b3-20472d79a220\") "
Sep 17 21:12:10.929301 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.929275    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "77d635a8-3d5f-4336-a9b3-20472d79a220" (UID: "77d635a8-3d5f-4336-a9b3-20472d79a220"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:12:10.931118 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.931089    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m" (OuterVolumeSpecName: "kube-api-access-2g46m") pod "77d635a8-3d5f-4336-a9b3-20472d79a220" (UID: "77d635a8-3d5f-4336-a9b3-20472d79a220"). InnerVolumeSpecName "kube-api-access-2g46m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:12:10.931228 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:10.931133    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "77d635a8-3d5f-4336-a9b3-20472d79a220" (UID: "77d635a8-3d5f-4336-a9b3-20472d79a220"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:12:11.030467 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.030409    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:12:11.030467 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.030465    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g46m\" (UniqueName: \"kubernetes.io/projected/77d635a8-3d5f-4336-a9b3-20472d79a220-kube-api-access-2g46m\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:12:11.030669 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.030478    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/77d635a8-3d5f-4336-a9b3-20472d79a220-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:12:11.663729 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.663692    2587 generic.go:356] "Generic (PLEG): container finished" podID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerID="0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b" exitCode=0
Sep 17 21:12:11.664102 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.663750    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"
Sep 17 21:12:11.664102 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.663776    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerDied","Data":"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"}
Sep 17 21:12:11.664102 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.663843    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9" event={"ID":"77d635a8-3d5f-4336-a9b3-20472d79a220","Type":"ContainerDied","Data":"7654c75fc8a22d25413aa4a986e1eda0fe5b57e35d8222bb08c02ababe31ef0a"}
Sep 17 21:12:11.664102 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.663872    2587 scope.go:117] "RemoveContainer" containerID="eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"
Sep 17 21:12:11.671606 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.671594    2587 scope.go:117] "RemoveContainer" containerID="0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"
Sep 17 21:12:11.682571 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.682539    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:12:11.682799 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.682733    2587 scope.go:117] "RemoveContainer" containerID="eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"
Sep 17 21:12:11.683040 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:12:11.683020    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51\": container with ID starting with eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51 not found: ID does not exist" containerID="eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"
Sep 17 21:12:11.683085 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.683048    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51"} err="failed to get container status \"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51\": rpc error: code = NotFound desc = could not find container \"eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51\": container with ID starting with eb082ef8b9ff39225e44cd99074e9b241a82dbe1f523c3b121e1cb610402be51 not found: ID does not exist"
Sep 17 21:12:11.683085 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.683067    2587 scope.go:117] "RemoveContainer" containerID="0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"
Sep 17 21:12:11.683350 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:12:11.683329    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b\": container with ID starting with 0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b not found: ID does not exist" containerID="0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"
Sep 17 21:12:11.683389 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.683354    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b"} err="failed to get container status \"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b\": rpc error: code = NotFound desc = could not find container \"0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b\": container with ID starting with 0caccff71ff6f10d3c006a00a49a505c0f999b7f292de5d378b88f6825181a9b not found: ID does not exist"
Sep 17 21:12:11.685711 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:11.685688    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-62e6f-predictor-6984c9779b-6qqq9"]
Sep 17 21:12:13.561744 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:13.561712    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" path="/var/lib/kubelet/pods/77d635a8-3d5f-4336-a9b3-20472d79a220/volumes"
Sep 17 21:12:15.663017 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:15.662983    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:12:15.663473 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:15.663443    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:19.676020 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:19.675988    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:20.415257 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:20.415215    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:12:25.663559 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:25.663519    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:35.663597 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:35.663555    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:45.664349 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:45.664299    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:12:55.663887 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:12:55.663842    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:13:05.663982 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:13:05.663952    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:13:17.237394 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:13:17.237316    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:17.416519 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:13:17.416485    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:13:25.666907 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:13:25.666877    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:17:16.245654 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:17:16.245613    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:18:48.144547 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:18:48.144509    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:34.910567 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:34.910532    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.036414 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.036370    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.043141 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.043114    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:20:35.043374 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.043346    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container" containerID="cri-o://a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613" gracePeriod=30
Sep 17 21:20:35.043486 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.043370    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kube-rbac-proxy" containerID="cri-o://609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390" gracePeriod=30
Sep 17 21:20:35.116610 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.116565    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.216303 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.216268    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.255958 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.255924    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:20:35.256224 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256213    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container"
Sep 17 21:20:35.256266 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256225    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container"
Sep 17 21:20:35.256266 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256241    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kube-rbac-proxy"
Sep 17 21:20:35.256266 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256247    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kube-rbac-proxy"
Sep 17 21:20:35.256359 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256297    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kserve-container"
Sep 17 21:20:35.256359 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.256305    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="77d635a8-3d5f-4336-a9b3-20472d79a220" containerName="kube-rbac-proxy"
Sep 17 21:20:35.259224 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.259206    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.262060 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.262041    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d3600-predictor-serving-cert\""
Sep 17 21:20:35.262279 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.262264    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-d3600-kube-rbac-proxy-sar-config\""
Sep 17 21:20:35.272769 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.272744    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:20:35.308286 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.308264    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.355560 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.355529    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.355676 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.355586    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rqjv\" (UniqueName: \"kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.355676 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.355644    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.456285 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.456252    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rqjv\" (UniqueName: \"kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.456478 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.456299    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.456478 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.456334    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.456995 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.456974    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.458738 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.458716    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.467298 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.467241    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rqjv\" (UniqueName: \"kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv\") pod \"error-404-isvc-d3600-predictor-5bc9bb5464-rns9p\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") " pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.512493 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.512413    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:35.525989 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.525965    2587 generic.go:356] "Generic (PLEG): container finished" podID="5b456083-3b26-453e-b265-f7bc0ba97343" containerID="609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390" exitCode=2
Sep 17 21:20:35.526095 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.526008    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerDied","Data":"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"}
Sep 17 21:20:35.568832 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.568801    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:35.841452 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:35.841359    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:20:35.845206 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:20:35.845177    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97d06122_12e7_41b2_ae57_57fe0151fe8d.slice/crio-37489d6c5614fe437a863bc8668c997b0f00b9cff38741f74c2db086cc26a81e WatchSource:0}: Error finding container 37489d6c5614fe437a863bc8668c997b0f00b9cff38741f74c2db086cc26a81e: Status 404 returned error can't find the container with id 37489d6c5614fe437a863bc8668c997b0f00b9cff38741f74c2db086cc26a81e
Sep 17 21:20:36.123492 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.123380    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:36.532014 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.531977    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerStarted","Data":"1ab5fab2e18ec9f7344d6d74328b980c8ae97978304e65d2cd4434f68f210b7d"}
Sep 17 21:20:36.532180 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.532034    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerStarted","Data":"f4c26027c713e4564847428fbb09969dc33b45909e4380de3182a8ebe2ed4994"}
Sep 17 21:20:36.532180 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.532057    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerStarted","Data":"37489d6c5614fe437a863bc8668c997b0f00b9cff38741f74c2db086cc26a81e"}
Sep 17 21:20:36.532180 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.532108    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:36.562076 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:36.562024    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podStartSLOduration=1.562002912 podStartE2EDuration="1.562002912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:20:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:20:36.561708191 +0000 UTC m=+1877.458096288" watchObservedRunningTime="2026-09-17 21:20:36.562002912 +0000 UTC m=+1877.458391017"
Sep 17 21:20:37.535701 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:37.535664    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:37.536958 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:37.536930    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:20:38.484196 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.484171    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:20:38.540858 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.540821    2587 generic.go:356] "Generic (PLEG): container finished" podID="5b456083-3b26-453e-b265-f7bc0ba97343" containerID="a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613" exitCode=0
Sep 17 21:20:38.541291 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.540920    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"
Sep 17 21:20:38.541291 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.540929    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerDied","Data":"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"}
Sep 17 21:20:38.541291 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.541122    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf" event={"ID":"5b456083-3b26-453e-b265-f7bc0ba97343","Type":"ContainerDied","Data":"d52114c0dcea754dc823d2743d381e3abfd66e8a10f89c3aa40bb1ca0f42d927"}
Sep 17 21:20:38.541291 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.541143    2587 scope.go:117] "RemoveContainer" containerID="609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"
Sep 17 21:20:38.541553 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.541408    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:20:38.549730 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.549703    2587 scope.go:117] "RemoveContainer" containerID="a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"
Sep 17 21:20:38.558222 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.558203    2587 scope.go:117] "RemoveContainer" containerID="609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"
Sep 17 21:20:38.558532 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:20:38.558506    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390\": container with ID starting with 609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390 not found: ID does not exist" containerID="609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"
Sep 17 21:20:38.558625 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.558543    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390"} err="failed to get container status \"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390\": rpc error: code = NotFound desc = could not find container \"609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390\": container with ID starting with 609abdf253fff07208aac186c1cb89de10134a96994aee8de1c540f018236390 not found: ID does not exist"
Sep 17 21:20:38.558625 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.558562    2587 scope.go:117] "RemoveContainer" containerID="a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"
Sep 17 21:20:38.558810 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:20:38.558792    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613\": container with ID starting with a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613 not found: ID does not exist" containerID="a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"
Sep 17 21:20:38.558859 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.558819    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613"} err="failed to get container status \"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613\": rpc error: code = NotFound desc = could not find container \"a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613\": container with ID starting with a0e7b06699828b12ecc7d59b1d1abad51b8b203174ceef2a364b75b2439f1613 not found: ID does not exist"
Sep 17 21:20:38.581274 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.581224    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wrqj\" (UniqueName: \"kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj\") pod \"5b456083-3b26-453e-b265-f7bc0ba97343\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") "
Sep 17 21:20:38.581360 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.581299    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") pod \"5b456083-3b26-453e-b265-f7bc0ba97343\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") "
Sep 17 21:20:38.581449 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.581405    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config\") pod \"5b456083-3b26-453e-b265-f7bc0ba97343\" (UID: \"5b456083-3b26-453e-b265-f7bc0ba97343\") "
Sep 17 21:20:38.581783 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.581757    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "5b456083-3b26-453e-b265-f7bc0ba97343" (UID: "5b456083-3b26-453e-b265-f7bc0ba97343"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:20:38.583243 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.583223    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5b456083-3b26-453e-b265-f7bc0ba97343" (UID: "5b456083-3b26-453e-b265-f7bc0ba97343"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:20:38.583382 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.583335    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj" (OuterVolumeSpecName: "kube-api-access-6wrqj") pod "5b456083-3b26-453e-b265-f7bc0ba97343" (UID: "5b456083-3b26-453e-b265-f7bc0ba97343"). InnerVolumeSpecName "kube-api-access-6wrqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:20:38.682571 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.682538    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/5b456083-3b26-453e-b265-f7bc0ba97343-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:20:38.682571 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.682566    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wrqj\" (UniqueName: \"kubernetes.io/projected/5b456083-3b26-453e-b265-f7bc0ba97343-kube-api-access-6wrqj\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:20:38.682571 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.682576    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5b456083-3b26-453e-b265-f7bc0ba97343-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:20:38.870965 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.870934    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:20:38.874927 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:38.874897    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-31e6a-predictor-764bc8c74b-kd5vf"]
Sep 17 21:20:39.561262 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:39.561226    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" path="/var/lib/kubelet/pods/5b456083-3b26-453e-b265-f7bc0ba97343/volumes"
Sep 17 21:20:43.545456 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:43.545364    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:20:43.545820 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:43.545795    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:20:53.546472 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:53.546409    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:20:57.279782 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:57.279749    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:20:57.373182 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:20:57.373147    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:03.546595 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:03.546546    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:21:13.546162 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:13.546123    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:21:23.546334 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:23.546289    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.26:8080: connect: connection refused"
Sep 17 21:21:32.286101 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.286062    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:32.383869 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.383827    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:32.397702 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.397666    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:21:32.397960 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.397930    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" containerID="cri-o://fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033" gracePeriod=30
Sep 17 21:21:32.398019 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.397997    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kube-rbac-proxy" containerID="cri-o://b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5" gracePeriod=30
Sep 17 21:21:32.453695 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.453665    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:32.530499 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.530470    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:32.602082 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602003    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:21:32.602280 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602262    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:32.602459 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602439    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kube-rbac-proxy"
Sep 17 21:21:32.602550 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602461    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kube-rbac-proxy"
Sep 17 21:21:32.602550 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602474    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container"
Sep 17 21:21:32.602550 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602481    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container"
Sep 17 21:21:32.602715 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602558    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kserve-container"
Sep 17 21:21:32.602715 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.602574    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b456083-3b26-453e-b265-f7bc0ba97343" containerName="kube-rbac-proxy"
Sep 17 21:21:32.605568 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.605550    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.608334 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.608315    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ccd0-predictor-serving-cert\""
Sep 17 21:21:32.608509 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.608492    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-3ccd0-kube-rbac-proxy-sar-config\""
Sep 17 21:21:32.619966 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.619940    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:21:32.700853 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.700819    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gcq7\" (UniqueName: \"kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.701010 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.700871    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.701010 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.700921    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.738483 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.738454    2587 generic.go:356] "Generic (PLEG): container finished" podID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerID="b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5" exitCode=2
Sep 17 21:21:32.738629 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.738505    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerDied","Data":"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"}
Sep 17 21:21:32.801864 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.801831    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gcq7\" (UniqueName: \"kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.802013 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.801875    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.802013 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.801900    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.802013 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:32.802010    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-serving-cert: secret "error-404-isvc-3ccd0-predictor-serving-cert" not found
Sep 17 21:21:32.802155 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:32.802061    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls podName:f610db45-c15c-4de5-b3b8-45a836aa7f0c nodeName:}" failed. No retries permitted until 2026-09-17 21:21:33.302042172 +0000 UTC m=+1934.198430243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls") pod "error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" (UID: "f610db45-c15c-4de5-b3b8-45a836aa7f0c") : secret "error-404-isvc-3ccd0-predictor-serving-cert" not found
Sep 17 21:21:32.802614 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.802591    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.810993 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.810968    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gcq7\" (UniqueName: \"kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:32.984720 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:32.984686    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:33.304967 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:33.304883    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:33.307241 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:33.307217    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") pod \"error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") " pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:33.515776 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:33.515704    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:33.547357 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:33.547330    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:21:33.790473 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:33.790443    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:21:33.793737 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:21:33.793706    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf610db45_c15c_4de5_b3b8_45a836aa7f0c.slice/crio-ffce0e1f0453e4387e689d8c6b6c07718474a1be9d4e94257f4e78a9ffed4907 WatchSource:0}: Error finding container ffce0e1f0453e4387e689d8c6b6c07718474a1be9d4e94257f4e78a9ffed4907: Status 404 returned error can't find the container with id ffce0e1f0453e4387e689d8c6b6c07718474a1be9d4e94257f4e78a9ffed4907
Sep 17 21:21:34.748419 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.748381    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerStarted","Data":"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"}
Sep 17 21:21:34.748870 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.748478    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerStarted","Data":"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"}
Sep 17 21:21:34.748870 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.748590    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:34.748870 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.748611    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:34.748870 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.748630    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerStarted","Data":"ffce0e1f0453e4387e689d8c6b6c07718474a1be9d4e94257f4e78a9ffed4907"}
Sep 17 21:21:34.750010 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.749982    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:21:34.771146 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:34.771101    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podStartSLOduration=2.771087314 podStartE2EDuration="2.771087314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:21:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:21:34.769197774 +0000 UTC m=+1935.665585869" watchObservedRunningTime="2026-09-17 21:21:34.771087314 +0000 UTC m=+1935.667475408"
Sep 17 21:21:35.658832 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:35.658791    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.25:8643/healthz\": dial tcp 10.133.0.25:8643: connect: connection refused"
Sep 17 21:21:35.664130 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:35.664106    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.25:8080: connect: connection refused"
Sep 17 21:21:35.752934 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:35.752902    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:21:35.937220 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:35.937196    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:21:36.024869 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.024837    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svllc\" (UniqueName: \"kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc\") pod \"852e4935-ae82-4b04-96c6-2a62961ab4b7\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") "
Sep 17 21:21:36.025049 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.024879    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config\") pod \"852e4935-ae82-4b04-96c6-2a62961ab4b7\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") "
Sep 17 21:21:36.025049 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.024927    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls\") pod \"852e4935-ae82-4b04-96c6-2a62961ab4b7\" (UID: \"852e4935-ae82-4b04-96c6-2a62961ab4b7\") "
Sep 17 21:21:36.025219 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.025194    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "852e4935-ae82-4b04-96c6-2a62961ab4b7" (UID: "852e4935-ae82-4b04-96c6-2a62961ab4b7"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:21:36.026917 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.026892    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc" (OuterVolumeSpecName: "kube-api-access-svllc") pod "852e4935-ae82-4b04-96c6-2a62961ab4b7" (UID: "852e4935-ae82-4b04-96c6-2a62961ab4b7"). InnerVolumeSpecName "kube-api-access-svllc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:21:36.027026 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.026989    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "852e4935-ae82-4b04-96c6-2a62961ab4b7" (UID: "852e4935-ae82-4b04-96c6-2a62961ab4b7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:21:36.126057 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.126020    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/852e4935-ae82-4b04-96c6-2a62961ab4b7-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:21:36.126057 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.126051    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svllc\" (UniqueName: \"kubernetes.io/projected/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-api-access-svllc\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:21:36.126057 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.126061    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/852e4935-ae82-4b04-96c6-2a62961ab4b7-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:21:36.758137 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.758104    2587 generic.go:356] "Generic (PLEG): container finished" podID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerID="fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033" exitCode=0
Sep 17 21:21:36.758137 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.758141    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerDied","Data":"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"}
Sep 17 21:21:36.758574 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.758150    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"
Sep 17 21:21:36.758574 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.758190    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b" event={"ID":"852e4935-ae82-4b04-96c6-2a62961ab4b7","Type":"ContainerDied","Data":"ecabc0781e22a7fcaa63550f6f9b804b0f79e2eaf1c5cb2edc41bfc27a0143ec"}
Sep 17 21:21:36.758574 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.758214    2587 scope.go:117] "RemoveContainer" containerID="b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"
Sep 17 21:21:36.766491 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.766472    2587 scope.go:117] "RemoveContainer" containerID="fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"
Sep 17 21:21:36.773355 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.773340    2587 scope.go:117] "RemoveContainer" containerID="b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"
Sep 17 21:21:36.773629 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:36.773612    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5\": container with ID starting with b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5 not found: ID does not exist" containerID="b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"
Sep 17 21:21:36.773682 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.773639    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5"} err="failed to get container status \"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5\": rpc error: code = NotFound desc = could not find container \"b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5\": container with ID starting with b546c7027e49583205fe3c4d0960c44f6fdfb5b30ea660507221f720e22209d5 not found: ID does not exist"
Sep 17 21:21:36.773682 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.773656    2587 scope.go:117] "RemoveContainer" containerID="fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"
Sep 17 21:21:36.773892 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:36.773871    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033\": container with ID starting with fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033 not found: ID does not exist" containerID="fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"
Sep 17 21:21:36.773944 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.773902    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033"} err="failed to get container status \"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033\": rpc error: code = NotFound desc = could not find container \"fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033\": container with ID starting with fac11caba3832b7856620dbbbc85fa6a8f461b13b18489aff396dad7f35d1033 not found: ID does not exist"
Sep 17 21:21:36.781707 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.781682    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:21:36.786143 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:36.786118    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-b5a48-predictor-5bb6b4f9f8-4b79b"]
Sep 17 21:21:37.562046 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:37.562013    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" path="/var/lib/kubelet/pods/852e4935-ae82-4b04-96c6-2a62961ab4b7/volumes"
Sep 17 21:21:40.757735 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:40.757706    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:21:40.758296 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:40.758270    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:21:45.206021 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:45.205996    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:45.294647 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:45.294613    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:50.759194 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:50.759151    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:21:56.232323 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.232285    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.372438 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.372392    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:21:56.372717 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.372690    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container" containerID="cri-o://f4c26027c713e4564847428fbb09969dc33b45909e4380de3182a8ebe2ed4994" gracePeriod=30
Sep 17 21:21:56.372837 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.372718    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kube-rbac-proxy" containerID="cri-o://1ab5fab2e18ec9f7344d6d74328b980c8ae97978304e65d2cd4434f68f210b7d" gracePeriod=30
Sep 17 21:21:56.373819 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.373790    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.486505 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.486417    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.584184 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.584152    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.626552 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626516    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:21:56.626843 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626831    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container"
Sep 17 21:21:56.626887 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626845    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container"
Sep 17 21:21:56.626887 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626857    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kube-rbac-proxy"
Sep 17 21:21:56.626887 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626864    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kube-rbac-proxy"
Sep 17 21:21:56.626990 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626915    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kserve-container"
Sep 17 21:21:56.626990 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.626924    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="852e4935-ae82-4b04-96c6-2a62961ab4b7" containerName="kube-rbac-proxy"
Sep 17 21:21:56.629776 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.629755    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.632662 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.632641    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a1ff2-predictor-serving-cert\""
Sep 17 21:21:56.632764 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.632641    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"error-404-isvc-a1ff2-kube-rbac-proxy-sar-config\""
Sep 17 21:21:56.642162 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.642136    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:21:56.676098 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.676066    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.680191 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.680168    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.680307 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.680295    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txjqx\" (UniqueName: \"kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.680408 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.680392    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.781857 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.781778    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txjqx\" (UniqueName: \"kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.781857 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.781827    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.782062 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.781876    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.782133 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:56.782120    2587 secret.go:189] Couldn't get secret kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-serving-cert: secret "error-404-isvc-a1ff2-predictor-serving-cert" not found
Sep 17 21:21:56.782195 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:21:56.782178    2587 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls podName:97fdc758-f620-4dea-b37e-7df3da9aa457 nodeName:}" failed. No retries permitted until 2026-09-17 21:21:57.282161623 +0000 UTC m=+1958.178549696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls") pod "error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" (UID: "97fdc758-f620-4dea-b37e-7df3da9aa457") : secret "error-404-isvc-a1ff2-predictor-serving-cert" not found
Sep 17 21:21:56.782550 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.782528    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.790882 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.790860    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txjqx\" (UniqueName: \"kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:56.830481 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.830443    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:56.836139 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.836109    2587 generic.go:356] "Generic (PLEG): container finished" podID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerID="1ab5fab2e18ec9f7344d6d74328b980c8ae97978304e65d2cd4434f68f210b7d" exitCode=2
Sep 17 21:21:56.836283 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.836183    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerDied","Data":"1ab5fab2e18ec9f7344d6d74328b980c8ae97978304e65d2cd4434f68f210b7d"}
Sep 17 21:21:56.998681 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:56.998644    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:21:57.285708 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:57.285671    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:57.288016 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:57.287985    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") pod \"error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") " pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:57.539860 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:57.539754    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:57.818298 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:57.818211    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:21:57.821285 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:21:57.821253    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97fdc758_f620_4dea_b37e_7df3da9aa457.slice/crio-ed5f6bc7f8a28b936347a1ce8a38877ea5fb21b610948b006044aa8bf835c465 WatchSource:0}: Error finding container ed5f6bc7f8a28b936347a1ce8a38877ea5fb21b610948b006044aa8bf835c465: Status 404 returned error can't find the container with id ed5f6bc7f8a28b936347a1ce8a38877ea5fb21b610948b006044aa8bf835c465
Sep 17 21:21:57.841636 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:57.841611    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerStarted","Data":"ed5f6bc7f8a28b936347a1ce8a38877ea5fb21b610948b006044aa8bf835c465"}
Sep 17 21:21:58.541435 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.541367    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.26:8643/healthz\": dial tcp 10.133.0.26:8643: connect: connection refused"
Sep 17 21:21:58.847865 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.847769    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerStarted","Data":"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"}
Sep 17 21:21:58.847865 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.847852    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerStarted","Data":"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"}
Sep 17 21:21:58.848080 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.847876    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:58.848080 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.847899    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:21:58.848989 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.848964    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:21:58.868721 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:58.868675    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podStartSLOduration=2.868661294 podStartE2EDuration="2.868661294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:21:58.867175548 +0000 UTC m=+1959.763563653" watchObservedRunningTime="2026-09-17 21:21:58.868661294 +0000 UTC m=+1959.765049387"
Sep 17 21:21:59.854052 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:59.854022    2587 generic.go:356] "Generic (PLEG): container finished" podID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerID="f4c26027c713e4564847428fbb09969dc33b45909e4380de3182a8ebe2ed4994" exitCode=0
Sep 17 21:21:59.854468 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:59.854095    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerDied","Data":"f4c26027c713e4564847428fbb09969dc33b45909e4380de3182a8ebe2ed4994"}
Sep 17 21:21:59.854542 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:59.854491    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:21:59.908236 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:21:59.908213    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:22:00.003733 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.003646    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config\") pod \"97d06122-12e7-41b2-ae57-57fe0151fe8d\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") "
Sep 17 21:22:00.003733 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.003680    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rqjv\" (UniqueName: \"kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv\") pod \"97d06122-12e7-41b2-ae57-57fe0151fe8d\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") "
Sep 17 21:22:00.003733 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.003733    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls\") pod \"97d06122-12e7-41b2-ae57-57fe0151fe8d\" (UID: \"97d06122-12e7-41b2-ae57-57fe0151fe8d\") "
Sep 17 21:22:00.004004 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.003977    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "97d06122-12e7-41b2-ae57-57fe0151fe8d" (UID: "97d06122-12e7-41b2-ae57-57fe0151fe8d"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:22:00.005861 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.005838    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv" (OuterVolumeSpecName: "kube-api-access-9rqjv") pod "97d06122-12e7-41b2-ae57-57fe0151fe8d" (UID: "97d06122-12e7-41b2-ae57-57fe0151fe8d"). InnerVolumeSpecName "kube-api-access-9rqjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:22:00.005965 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.005905    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "97d06122-12e7-41b2-ae57-57fe0151fe8d" (UID: "97d06122-12e7-41b2-ae57-57fe0151fe8d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:22:00.104545 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.104504    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97d06122-12e7-41b2-ae57-57fe0151fe8d-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:22:00.104545 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.104536    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:22:00.104545 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.104546    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rqjv\" (UniqueName: \"kubernetes.io/projected/97d06122-12e7-41b2-ae57-57fe0151fe8d-kube-api-access-9rqjv\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:22:00.758685 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.758643    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:22:00.859684 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.859656    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"
Sep 17 21:22:00.859684 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.859678    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p" event={"ID":"97d06122-12e7-41b2-ae57-57fe0151fe8d","Type":"ContainerDied","Data":"37489d6c5614fe437a863bc8668c997b0f00b9cff38741f74c2db086cc26a81e"}
Sep 17 21:22:00.860265 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.859747    2587 scope.go:117] "RemoveContainer" containerID="1ab5fab2e18ec9f7344d6d74328b980c8ae97978304e65d2cd4434f68f210b7d"
Sep 17 21:22:00.869018 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.868908    2587 scope.go:117] "RemoveContainer" containerID="f4c26027c713e4564847428fbb09969dc33b45909e4380de3182a8ebe2ed4994"
Sep 17 21:22:00.882175 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.882153    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:22:00.886520 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:00.886495    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-d3600-predictor-5bc9bb5464-rns9p"]
Sep 17 21:22:01.561779 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:01.561738    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" path="/var/lib/kubelet/pods/97d06122-12e7-41b2-ae57-57fe0151fe8d/volumes"
Sep 17 21:22:04.859684 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:04.859655    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:22:04.860146 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:04.860118    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:22:10.758839 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:10.758798    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:22:14.860799 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:14.860761    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:22:20.758851 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:20.758809    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.27:8080: connect: connection refused"
Sep 17 21:22:24.860619 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:24.860574    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:22:30.759222 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:30.759190    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:22:34.860719 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:34.860677    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:22:42.539917 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:42.539880    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:22:44.171065 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:44.171028    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:22:44.860344 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:44.860305    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:22:54.861125 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:22:54.861089    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:23:06.441580 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:23:06.441544    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:23:06.732731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:23:06.732700    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:27:50.578357 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:27:50.578319    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:31:21.706500 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.706377    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:31:21.819639 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.819602    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:31:21.834151 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.834111    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:31:21.834417 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.834376    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" containerID="cri-o://b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded" gracePeriod=30
Sep 17 21:31:21.834588 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.834413    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kube-rbac-proxy" containerID="cri-o://8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304" gracePeriod=30
Sep 17 21:31:21.899019 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:21.898976    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:31:22.946844 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:22.946808    2587 generic.go:356] "Generic (PLEG): container finished" podID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerID="8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304" exitCode=2
Sep 17 21:31:22.947216 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:22.946882    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerDied","Data":"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"}
Sep 17 21:31:24.855119 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:24.855078    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.28:8643/healthz\": dial tcp 10.133.0.28:8643: connect: connection refused"
Sep 17 21:31:24.860487 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:24.860459    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.28:8080: connect: connection refused"
Sep 17 21:31:25.186194 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.186171    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:31:25.219488 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.219462    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") pod \"97fdc758-f620-4dea-b37e-7df3da9aa457\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") "
Sep 17 21:31:25.219639 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.219503    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config\") pod \"97fdc758-f620-4dea-b37e-7df3da9aa457\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") "
Sep 17 21:31:25.219639 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.219569    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txjqx\" (UniqueName: \"kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx\") pod \"97fdc758-f620-4dea-b37e-7df3da9aa457\" (UID: \"97fdc758-f620-4dea-b37e-7df3da9aa457\") "
Sep 17 21:31:25.219899 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.219876    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "97fdc758-f620-4dea-b37e-7df3da9aa457" (UID: "97fdc758-f620-4dea-b37e-7df3da9aa457"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:31:25.221460 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.221434    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "97fdc758-f620-4dea-b37e-7df3da9aa457" (UID: "97fdc758-f620-4dea-b37e-7df3da9aa457"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:31:25.221577 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.221560    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx" (OuterVolumeSpecName: "kube-api-access-txjqx") pod "97fdc758-f620-4dea-b37e-7df3da9aa457" (UID: "97fdc758-f620-4dea-b37e-7df3da9aa457"). InnerVolumeSpecName "kube-api-access-txjqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:31:25.320506 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.320476    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/97fdc758-f620-4dea-b37e-7df3da9aa457-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:31:25.320506 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.320500    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:31:25.320506 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.320511    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txjqx\" (UniqueName: \"kubernetes.io/projected/97fdc758-f620-4dea-b37e-7df3da9aa457-kube-api-access-txjqx\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:31:25.958627 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.958594    2587 generic.go:356] "Generic (PLEG): container finished" podID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerID="b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded" exitCode=0
Sep 17 21:31:25.959001 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.958646    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"
Sep 17 21:31:25.959001 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.958676    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerDied","Data":"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"}
Sep 17 21:31:25.959001 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.958742    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj" event={"ID":"97fdc758-f620-4dea-b37e-7df3da9aa457","Type":"ContainerDied","Data":"ed5f6bc7f8a28b936347a1ce8a38877ea5fb21b610948b006044aa8bf835c465"}
Sep 17 21:31:25.959001 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.958768    2587 scope.go:117] "RemoveContainer" containerID="8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"
Sep 17 21:31:25.966899 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.966880    2587 scope.go:117] "RemoveContainer" containerID="b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"
Sep 17 21:31:25.974776 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.974736    2587 scope.go:117] "RemoveContainer" containerID="8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"
Sep 17 21:31:25.975101 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:31:25.975080    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304\": container with ID starting with 8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304 not found: ID does not exist" containerID="8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"
Sep 17 21:31:25.975174 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.975115    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304"} err="failed to get container status \"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304\": rpc error: code = NotFound desc = could not find container \"8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304\": container with ID starting with 8d7cc8ef74a270591728a4149c120454d71cea0e545a0af1655f6608e2eda304 not found: ID does not exist"
Sep 17 21:31:25.975174 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.975134    2587 scope.go:117] "RemoveContainer" containerID="b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"
Sep 17 21:31:25.975466 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:31:25.975443    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded\": container with ID starting with b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded not found: ID does not exist" containerID="b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"
Sep 17 21:31:25.975519 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.975475    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded"} err="failed to get container status \"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded\": rpc error: code = NotFound desc = could not find container \"b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded\": container with ID starting with b6c7f6d23ece61e6336fdbff6308a207be1f1b8425e92873a3be3bdd0fe90ded not found: ID does not exist"
Sep 17 21:31:25.976093 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.976074    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:31:25.978373 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:25.978350    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-a1ff2-predictor-6bd48997f4-pp8xj"]
Sep 17 21:31:27.562144 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:31:27.562109    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" path="/var/lib/kubelet/pods/97fdc758-f620-4dea-b37e-7df3da9aa457/volumes"
Sep 17 21:32:35.204637 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:32:35.204593    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:33:54.752947 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:33:54.752914    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:02.825510 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:02.825475    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:02.935530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:02.935487    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:39:02.935733 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:02.935704    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container" containerID="cri-o://b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e" gracePeriod=30
Sep 17 21:39:02.935800 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:02.935733    2587 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kube-rbac-proxy" containerID="cri-o://5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1" gracePeriod=30
Sep 17 21:39:03.603243 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:03.603206    2587 generic.go:356] "Generic (PLEG): container finished" podID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerID="5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1" exitCode=2
Sep 17 21:39:03.603413 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:03.603267    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerDied","Data":"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"}
Sep 17 21:39:04.095734 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:04.095691    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:05.757237 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:05.757192    2587 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.27:8643/healthz\": dial tcp 10.133.0.27:8643: connect: connection refused"
Sep 17 21:39:05.985013 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:05.984988    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:39:06.051864 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.051782    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") pod \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") "
Sep 17 21:39:06.051864 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.051832    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config\") pod \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") "
Sep 17 21:39:06.051864 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.051863    2587 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gcq7\" (UniqueName: \"kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7\") pod \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\" (UID: \"f610db45-c15c-4de5-b3b8-45a836aa7f0c\") "
Sep 17 21:39:06.052215 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.052188    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f610db45-c15c-4de5-b3b8-45a836aa7f0c" (UID: "f610db45-c15c-4de5-b3b8-45a836aa7f0c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 17 21:39:06.053871 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.053839    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f610db45-c15c-4de5-b3b8-45a836aa7f0c" (UID: "f610db45-c15c-4de5-b3b8-45a836aa7f0c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 17 21:39:06.053988 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.053967    2587 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7" (OuterVolumeSpecName: "kube-api-access-9gcq7") pod "f610db45-c15c-4de5-b3b8-45a836aa7f0c" (UID: "f610db45-c15c-4de5-b3b8-45a836aa7f0c"). InnerVolumeSpecName "kube-api-access-9gcq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 17 21:39:06.152530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.152496    2587 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gcq7\" (UniqueName: \"kubernetes.io/projected/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-api-access-9gcq7\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:39:06.152530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.152526    2587 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f610db45-c15c-4de5-b3b8-45a836aa7f0c-proxy-tls\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:39:06.152530 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.152537    2587 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f610db45-c15c-4de5-b3b8-45a836aa7f0c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-138-134.ec2.internal\" DevicePath \"\""
Sep 17 21:39:06.615308 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.615274    2587 generic.go:356] "Generic (PLEG): container finished" podID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerID="b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e" exitCode=0
Sep 17 21:39:06.615495 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.615329    2587 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"
Sep 17 21:39:06.615495 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.615351    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerDied","Data":"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"}
Sep 17 21:39:06.615495 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.615397    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl" event={"ID":"f610db45-c15c-4de5-b3b8-45a836aa7f0c","Type":"ContainerDied","Data":"ffce0e1f0453e4387e689d8c6b6c07718474a1be9d4e94257f4e78a9ffed4907"}
Sep 17 21:39:06.615495 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.615447    2587 scope.go:117] "RemoveContainer" containerID="5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"
Sep 17 21:39:06.623527 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.623514    2587 scope.go:117] "RemoveContainer" containerID="b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"
Sep 17 21:39:06.630489 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.630473    2587 scope.go:117] "RemoveContainer" containerID="5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"
Sep 17 21:39:06.630758 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:39:06.630740    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1\": container with ID starting with 5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1 not found: ID does not exist" containerID="5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"
Sep 17 21:39:06.630811 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.630769    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1"} err="failed to get container status \"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1\": rpc error: code = NotFound desc = could not find container \"5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1\": container with ID starting with 5b597572a6d50717fb44fbb4d64be0d120850c733561edb354f8942fb7c9dee1 not found: ID does not exist"
Sep 17 21:39:06.630811 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.630785    2587 scope.go:117] "RemoveContainer" containerID="b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"
Sep 17 21:39:06.631011 ip-10-0-138-134 kubenswrapper[2587]: E0917 21:39:06.630993    2587 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e\": container with ID starting with b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e not found: ID does not exist" containerID="b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"
Sep 17 21:39:06.631052 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.631016    2587 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e"} err="failed to get container status \"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e\": rpc error: code = NotFound desc = could not find container \"b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e\": container with ID starting with b605dc24567c767a09901f2b613887fe677181bb854d11aac9b3188d2b0f0d2e not found: ID does not exist"
Sep 17 21:39:06.636356 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.636335    2587 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:39:06.640274 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:06.640251    2587 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/error-404-isvc-3ccd0-predictor-5c789f8bd8-c7nfl"]
Sep 17 21:39:07.561778 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:07.561745    2587 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" path="/var/lib/kubelet/pods/f610db45-c15c-4de5-b3b8-45a836aa7f0c/volumes"
Sep 17 21:39:35.307846 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.307810    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:35.473127 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473087    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-44q44/must-gather-s26jn"]
Sep 17 21:39:35.473377 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473366    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473379    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473388    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473394    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473405    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473411    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473417    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473435    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473444    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container"
Sep 17 21:39:35.473451 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473449    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473460    2587 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473466    2587 state_mem.go:107] "Deleted CPUSet assignment" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473510    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473519    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473525    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kube-rbac-proxy"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473532    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="97d06122-12e7-41b2-ae57-57fe0151fe8d" containerName="kserve-container"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473537    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="97fdc758-f620-4dea-b37e-7df3da9aa457" containerName="kserve-container"
Sep 17 21:39:35.473731 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.473542    2587 memory_manager.go:368] "RemoveStaleState removing state" podUID="f610db45-c15c-4de5-b3b8-45a836aa7f0c" containerName="kserve-container"
Sep 17 21:39:35.476329 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.476312    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.480222 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.480198    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-44q44\"/\"kube-root-ca.crt\""
Sep 17 21:39:35.480365 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.480198    2587 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-44q44\"/\"default-dockercfg-rvlpj\""
Sep 17 21:39:35.480622 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.480602    2587 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-44q44\"/\"openshift-service-ca.crt\""
Sep 17 21:39:35.498189 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.498166    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44q44/must-gather-s26jn"]
Sep 17 21:39:35.514623 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.514585    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:35.587513 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.587443    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-must-gather-output\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.587513 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.587487    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-kube-api-access-2ckzf\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.612454 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.612416    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:35.688755 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.688723    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-must-gather-output\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.688887 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.688781    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-kube-api-access-2ckzf\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.689056 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.689038    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-must-gather-output\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.702611 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.702577    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ckzf\" (UniqueName: \"kubernetes.io/projected/faaef1d1-edeb-49ee-9e17-ee831d8d18d6-kube-api-access-2ckzf\") pod \"must-gather-s26jn\" (UID: \"faaef1d1-edeb-49ee-9e17-ee831d8d18d6\") " pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:35.785900 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:35.785854    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44q44/must-gather-s26jn"
Sep 17 21:39:36.055897 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:36.055875    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44q44/must-gather-s26jn"]
Sep 17 21:39:36.058090 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:39:36.058063    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfaaef1d1_edeb_49ee_9e17_ee831d8d18d6.slice/crio-b417f55de3c20fc75f2f9be7d507fefcce7f348438a7c64f53099d4ebc9e7ac5 WatchSource:0}: Error finding container b417f55de3c20fc75f2f9be7d507fefcce7f348438a7c64f53099d4ebc9e7ac5: Status 404 returned error can't find the container with id b417f55de3c20fc75f2f9be7d507fefcce7f348438a7c64f53099d4ebc9e7ac5
Sep 17 21:39:36.062418 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:36.062397    2587 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 17 21:39:36.722174 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:36.722138    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44q44/must-gather-s26jn" event={"ID":"faaef1d1-edeb-49ee-9e17-ee831d8d18d6","Type":"ContainerStarted","Data":"b417f55de3c20fc75f2f9be7d507fefcce7f348438a7c64f53099d4ebc9e7ac5"}
Sep 17 21:39:37.730694 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:37.730658    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44q44/must-gather-s26jn" event={"ID":"faaef1d1-edeb-49ee-9e17-ee831d8d18d6","Type":"ContainerStarted","Data":"f38e1e888533912589b1dc8953f5253f06b46638585f6e3dda61020c26f11294"}
Sep 17 21:39:37.731063 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:37.730702    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44q44/must-gather-s26jn" event={"ID":"faaef1d1-edeb-49ee-9e17-ee831d8d18d6","Type":"ContainerStarted","Data":"da7ba7e870acc32babc89524e72cafe0f4636c6292a395d9e27915518a500154"}
Sep 17 21:39:37.752397 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:37.752347    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-44q44/must-gather-s26jn" podStartSLOduration=1.999024607 podStartE2EDuration="2.752330454s" totalImagesPullingTime="753.305847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 21:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:39:37.750177263 +0000 UTC m=+3018.646565370" watchObservedRunningTime="2026-09-17 21:39:37.752330454 +0000 UTC m=+3018.648718573"
Sep 17 21:39:38.242759 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:38.242712    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fjsrk_658b0509-365d-4197-bb9e-682ffdccacb2/global-pull-secret-syncer/0.log"
Sep 17 21:39:38.364497 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:38.364466    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log"
Sep 17 21:39:38.445382 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:38.445350    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-134.ec2.internal_54325897c1c76625ff4176982d832399/haproxy/0.log"
Sep 17 21:39:40.969400 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:40.969331    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:41.068235 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:41.068203    2587 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8zpxn_bfd56693-7499-45af-acb5-fac485d47c8f/konnectivity-agent/0.log" limit=1024
Sep 17 21:39:43.173721 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.173689    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-spldr_0f8ddb33-fb5f-4238-974e-2de5e4847d09/kube-state-metrics/0.log"
Sep 17 21:39:43.216879 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.216843    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-spldr_0f8ddb33-fb5f-4238-974e-2de5e4847d09/kube-rbac-proxy-main/0.log"
Sep 17 21:39:43.249869 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.249832    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-spldr_0f8ddb33-fb5f-4238-974e-2de5e4847d09/kube-rbac-proxy-self/0.log"
Sep 17 21:39:43.316325 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.316290    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7647ffd697-gjb2q_b9c95342-2ca7-4453-ab3d-44b73479128e/metrics-server/0.log"
Sep 17 21:39:43.613679 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.613650    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzph6_6d2724f0-7a6f-481a-af6a-91ab603599d8/node-exporter/0.log"
Sep 17 21:39:43.646811 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.646784    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzph6_6d2724f0-7a6f-481a-af6a-91ab603599d8/kube-rbac-proxy/0.log"
Sep 17 21:39:43.678808 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:43.678777    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzph6_6d2724f0-7a6f-481a-af6a-91ab603599d8/init-textfile/0.log"
Sep 17 21:39:45.801106 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.801070    2587 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"]
Sep 17 21:39:45.806742 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.806479    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.816356 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.816089    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"]
Sep 17 21:39:45.877725 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.877487    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-lib-modules\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.877725 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.877547    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-proc\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.877725 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.877606    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz2jk\" (UniqueName: \"kubernetes.io/projected/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-kube-api-access-pz2jk\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.877725 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.877646    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-podres\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.877725 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.877674    2587 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-sys\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978339 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978300    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz2jk\" (UniqueName: \"kubernetes.io/projected/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-kube-api-access-pz2jk\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978359    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-podres\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978392    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-sys\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978463    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-lib-modules\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978501    2587 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-proc\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978513    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-sys\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978537 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978511    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-podres\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978809 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978594    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-lib-modules\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.978809 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.978619    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-proc\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:45.986555 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:45.986527    2587 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz2jk\" (UniqueName: \"kubernetes.io/projected/75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4-kube-api-access-pz2jk\") pod \"perf-node-gather-daemonset-xs6zb\" (UID: \"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4\") " pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:46.122250 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.122167    2587 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:46.408505 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.408393    2587 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"]
Sep 17 21:39:46.411661 ip-10-0-138-134 kubenswrapper[2587]: W0917 21:39:46.411630    2587 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75eed97a_9098_4ffe_a4f6_33cbf8f7aaf4.slice/crio-ca9ac681be3304b5ff1d9689b7132266577ae586bfc7e2be545556495df2578d WatchSource:0}: Error finding container ca9ac681be3304b5ff1d9689b7132266577ae586bfc7e2be545556495df2578d: Status 404 returned error can't find the container with id ca9ac681be3304b5ff1d9689b7132266577ae586bfc7e2be545556495df2578d
Sep 17 21:39:46.781812 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.781769    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb" event={"ID":"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4","Type":"ContainerStarted","Data":"d3dde1467efb891427e45ac0e745b35073d5381c221eec1893c38a65fcc128e7"}
Sep 17 21:39:46.782013 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.781840    2587 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb" event={"ID":"75eed97a-9098-4ffe-a4f6-33cbf8f7aaf4","Type":"ContainerStarted","Data":"ca9ac681be3304b5ff1d9689b7132266577ae586bfc7e2be545556495df2578d"}
Sep 17 21:39:46.782013 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.781954    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:46.799672 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:46.799628    2587 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb" podStartSLOduration=1.7996094550000001 podStartE2EDuration="1.799609455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 21:39:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 21:39:46.799175115 +0000 UTC m=+3027.695563212" watchObservedRunningTime="2026-09-17 21:39:46.799609455 +0000 UTC m=+3027.695997551"
Sep 17 21:39:48.255595 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:48.255562    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brdvm_159ae6fc-327d-498f-b31e-94a7396a9abb/dns/0.log"
Sep 17 21:39:48.289058 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:48.289027    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-brdvm_159ae6fc-327d-498f-b31e-94a7396a9abb/kube-rbac-proxy/0.log"
Sep 17 21:39:48.388087 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:48.388059    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9nksj_602783c1-0d44-4f67-ab95-e2c99cd0fe21/dns-node-resolver/0.log"
Sep 17 21:39:48.938503 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:48.938475    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bn9d2_3c6387a1-2f78-4e35-a245-65dfe1c0dea5/node-ca/0.log"
Sep 17 21:39:49.722214 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:49.722192    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5995fd8d76-x6twc_3fda01da-95e1-47b6-b6f2-ad05635edf2b/router/0.log"
Sep 17 21:39:50.043294 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:50.043217    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2dpzv_2c352448-d5b6-4c8f-aeba-ead56fa5b354/serve-healthcheck-canary/0.log"
Sep 17 21:39:50.486502 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:50.486471    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfjlm_dd5781c3-0930-4d58-86b0-55ef0b39efe7/kube-rbac-proxy/0.log"
Sep 17 21:39:50.507871 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:50.507846    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfjlm_dd5781c3-0930-4d58-86b0-55ef0b39efe7/exporter/0.log"
Sep 17 21:39:50.529187 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:50.529160    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gfjlm_dd5781c3-0930-4d58-86b0-55ef0b39efe7/extractor/0.log"
Sep 17 21:39:52.795686 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:52.795657    2587 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-44q44/perf-node-gather-daemonset-xs6zb"
Sep 17 21:39:53.013547 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:53.013519    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-sh2f2_6659ae6b-d988-47cf-8939-bc838da02e53/manager/0.log"
Sep 17 21:39:53.133028 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:53.132955    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-g6768_2e5e9082-1c9f-4abd-9847-4ff84e1e8e82/manager/0.log"
Sep 17 21:39:57.650938 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:57.650909    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-tlhb4_470a5f8b-7034-4af0-a46d-6cde6a4a492f/migrator/0.log"
Sep 17 21:39:57.671780 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:57.671755    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-tlhb4_470a5f8b-7034-4af0-a46d-6cde6a4a492f/graceful-termination/0.log"
Sep 17 21:39:58.012539 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:58.012508    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-r8bgv_fbeaf482-76e0-4142-9f43-83b6e669c9c7/kube-storage-version-migrator-operator/1.log"
Sep 17 21:39:58.013400 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:58.013380    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-r8bgv_fbeaf482-76e0-4142-9f43-83b6e669c9c7/kube-storage-version-migrator-operator/0.log"
Sep 17 21:39:58.970944 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:58.970917    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/kube-multus-additional-cni-plugins/0.log"
Sep 17 21:39:58.991720 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:58.991675    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/egress-router-binary-copy/0.log"
Sep 17 21:39:59.013076 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.013029    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/cni-plugins/0.log"
Sep 17 21:39:59.036019 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.035998    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/bond-cni-plugin/0.log"
Sep 17 21:39:59.055721 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.055701    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/routeoverride-cni/0.log"
Sep 17 21:39:59.075260 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.075233    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/whereabouts-cni-bincopy/0.log"
Sep 17 21:39:59.109459 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.109436    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2m5mc_9a43e2b1-ae80-4ced-9db5-9a82f26503cb/whereabouts-cni/0.log"
Sep 17 21:39:59.439608 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.439536    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-b6fzd_9d00bb16-e138-4456-828c-916737f65dc4/kube-multus/0.log"
Sep 17 21:39:59.532638 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.532612    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ldldg_ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa/network-metrics-daemon/0.log"
Sep 17 21:39:59.553163 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:39:59.553140    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ldldg_ac1cee9d-3a8b-4c03-ab99-f2f5634f02aa/kube-rbac-proxy/0.log"
Sep 17 21:40:00.743221 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.743190    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/ovn-controller/0.log"
Sep 17 21:40:00.765616 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.765591    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/ovn-acl-logging/0.log"
Sep 17 21:40:00.789220 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.789198    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/kube-rbac-proxy-node/0.log"
Sep 17 21:40:00.810551 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.810505    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/kube-rbac-proxy-ovn-metrics/0.log"
Sep 17 21:40:00.829676 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.829657    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/northd/0.log"
Sep 17 21:40:00.853510 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.853491    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/nbdb/0.log"
Sep 17 21:40:00.876910 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.876891    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/sbdb/0.log"
Sep 17 21:40:00.923228 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:00.923207    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z9wf9_d0263dac-21f2-4cbf-a262-5aaff8eae17d/ovnkube-controller/0.log"
Sep 17 21:40:01.927332 ip-10-0-138-134 kubenswrapper[2587]: I0917 21:40:01.927303    2587 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-46x9r_d356f916-1ba5-463a-97ae-bc8030fc8d8f/network-check-target-container/0.log"