Apr 16 08:36:37.557048 ip-10-0-137-248 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Apr 16 08:36:37.557063 ip-10-0-137-248 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Apr 16 08:36:37.557073 ip-10-0-137-248 systemd[1]: kubelet.service: Failed with result 'resources'. Apr 16 08:36:37.557404 ip-10-0-137-248 systemd[1]: Failed to start Kubernetes Kubelet. Apr 16 08:36:47.801517 ip-10-0-137-248 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Apr 16 08:36:47.801538 ip-10-0-137-248 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 75f0c5acf60c4ae5bf22846125927187 -- Apr 16 08:38:56.505372 ip-10-0-137-248 systemd[1]: Starting Kubernetes Kubelet... Apr 16 08:38:57.018066 ip-10-0-137-248 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:38:57.018066 ip-10-0-137-248 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Apr 16 08:38:57.018066 ip-10-0-137-248 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:38:57.018727 ip-10-0-137-248 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Apr 16 08:38:57.018727 ip-10-0-137-248 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Apr 16 08:38:57.021795 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.021688 2569 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Apr 16 08:38:57.025379 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025357 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:38:57.025379 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025377 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:38:57.025379 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025381 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:38:57.025379 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025384 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:38:57.025379 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025387 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025391 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025393 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025397 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025400 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025402 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025406 2569 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025410 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025415 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025420 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025423 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025426 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025431 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025433 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025436 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025439 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025441 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025444 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025447 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:38:57.025566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025449 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025452 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025454 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025464 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025467 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025469 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025472 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025474 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025477 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025479 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025482 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025484 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025487 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025489 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025492 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025495 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025498 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025501 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025504 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025506 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:38:57.026028 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025509 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025511 2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025514 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025516 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025519 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025521 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025524 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025527 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025530 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025532 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025535 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025537 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025540 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025543 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025545 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025548 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025550 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025553 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025556 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025559 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:38:57.026568 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025562 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025564 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025567 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025569 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025572 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025574 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025577 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025581 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025585 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025587 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025590 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025592 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025595 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025598 2569 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025600 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025609 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025612 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025614 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025617 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025620 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:38:57.027040 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025622 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025625 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.025627 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026097 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026104 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026106 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026109 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026112 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026114 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026117 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026119 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026122 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026125 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026127 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026130 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026132 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026135 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026137 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026140 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026143 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:38:57.027529 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026146 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026148 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026151 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026156 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026160 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026163 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026166 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026175 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026178 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026180 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026183 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026185 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026188 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026190 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026193 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026195 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026198 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026200 2569 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026203 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:38:57.027999 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026205 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026208 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026210 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026213 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026216 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026219 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026221 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026225 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026228 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026231 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026233 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026236 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026239 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026242 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026245 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026247 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026249 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026252 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026255 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:38:57.028484 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026257 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026259 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026268 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026270 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026273 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026275 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026278 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026280 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026283 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026285 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026287 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026290 2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026292 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026294 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026297 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026299 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026301 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026305 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026308 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026310 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:38:57.028938 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026312 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026315 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026317 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026320 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026322 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026325 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026327 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026330 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026332 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026335 2569 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.026337 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.027978 2569 flags.go:64] FLAG: --address="0.0.0.0" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.027989 2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.027999 2569 flags.go:64] FLAG: --anonymous-auth="true" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028004 2569 flags.go:64] FLAG: --application-metrics-count-limit="100" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028015 2569 flags.go:64] FLAG: --authentication-token-webhook="false" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028019 2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028024 2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028028 2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028031 2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028034 2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028037 2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Apr 16 08:38:57.029443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028040 2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028043 2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028046 2569 flags.go:64] FLAG: --cgroup-root="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028068 2569 flags.go:64] FLAG: --cgroups-per-qos="true" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028071 2569 flags.go:64] FLAG: --client-ca-file="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028075 2569 flags.go:64] FLAG: --cloud-config="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028078 2569 flags.go:64] FLAG: --cloud-provider="external" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028080 2569 flags.go:64] FLAG: --cluster-dns="[]" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028088 2569 flags.go:64] FLAG: --cluster-domain="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028091 2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028094 2569 flags.go:64] FLAG: --config-dir="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028097 2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028100 2569 flags.go:64] FLAG: --container-log-max-files="5" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028105 2569 flags.go:64] FLAG: --container-log-max-size="10Mi" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028108 2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028111 2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028114 2569 flags.go:64] FLAG: --containerd-namespace="k8s.io" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028117 2569 flags.go:64] FLAG: --contention-profiling="false" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028120 2569 flags.go:64] FLAG: --cpu-cfs-quota="true" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028123 2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028126 2569 flags.go:64] FLAG: --cpu-manager-policy="none" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028129 2569 flags.go:64] FLAG: --cpu-manager-policy-options="" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028133 2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028136 2569 flags.go:64] FLAG: --enable-controller-attach-detach="true" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028139 2569 flags.go:64] FLAG: --enable-debugging-handlers="true" Apr 16 08:38:57.029982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028142 2569 flags.go:64] FLAG: --enable-load-reader="false" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028151 2569 flags.go:64] FLAG: --enable-server="true" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028154 2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028161 2569 flags.go:64] FLAG: --event-burst="100" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028164 2569 flags.go:64] FLAG: --event-qps="50" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028167 2569 flags.go:64] FLAG: --event-storage-age-limit="default=0" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028170 2569 flags.go:64] FLAG: --event-storage-event-limit="default=0" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028173 2569 flags.go:64] FLAG: --eviction-hard="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028177 2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028180 2569 flags.go:64] FLAG: --eviction-minimum-reclaim="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028184 2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028186 2569 flags.go:64] FLAG: --eviction-soft="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028195 2569 flags.go:64] FLAG: --eviction-soft-grace-period="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028197 2569 flags.go:64] FLAG: --exit-on-lock-contention="false" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028200 2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028203 2569 flags.go:64] FLAG: --experimental-mounter-path="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028206 2569 flags.go:64] FLAG: --fail-cgroupv1="false" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028209 2569 flags.go:64] FLAG: --fail-swap-on="true" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028211 2569 flags.go:64] FLAG: --feature-gates="" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028216 2569 flags.go:64] FLAG: --file-check-frequency="20s" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028219 2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028221 2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028225 2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028228 2569 flags.go:64] FLAG: --healthz-port="10248" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028231 2569 flags.go:64] FLAG: --help="false" Apr 16 08:38:57.030615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028234 2569 flags.go:64] FLAG: --hostname-override="ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028237 2569 flags.go:64] FLAG: --housekeeping-interval="10s" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028240 2569 flags.go:64] FLAG: --http-check-frequency="20s" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028243 2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028246 2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028249 2569 flags.go:64] FLAG: --image-gc-high-threshold="85" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028252 2569 flags.go:64] FLAG: --image-gc-low-threshold="80" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028255 2569 flags.go:64] FLAG: --image-service-endpoint="" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028257 2569 flags.go:64] FLAG: --kernel-memcg-notification="false" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028266 2569 flags.go:64] FLAG: --kube-api-burst="100" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028269 2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028272 2569 flags.go:64] FLAG: --kube-api-qps="50" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028274 2569 flags.go:64] FLAG: --kube-reserved="" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028277 2569 flags.go:64] FLAG: --kube-reserved-cgroup="" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028280 2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028283 2569 flags.go:64] FLAG: --kubelet-cgroups="" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028285 2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028289 2569 flags.go:64] FLAG: --lock-file="" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028291 2569 flags.go:64] FLAG: --log-cadvisor-usage="false" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028295 2569 flags.go:64] FLAG: --log-flush-frequency="5s" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028298 2569 flags.go:64] FLAG: --log-json-info-buffer-size="0" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028304 2569 flags.go:64] FLAG: --log-json-split-stream="false" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028307 2569 flags.go:64] FLAG: --log-text-info-buffer-size="0" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028309 2569 flags.go:64] FLAG: --log-text-split-stream="false" Apr 16 08:38:57.031239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028312 2569 flags.go:64] FLAG: --logging-format="text" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028316 2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028319 2569 flags.go:64] FLAG: --make-iptables-util-chains="true" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028327 2569 flags.go:64] FLAG: --manifest-url="" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028331 2569 flags.go:64] FLAG: --manifest-url-header="" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028335 2569 flags.go:64] FLAG: --max-housekeeping-interval="15s" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028338 2569 flags.go:64] FLAG: --max-open-files="1000000" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028342 2569 flags.go:64] FLAG: --max-pods="110" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028345 2569 flags.go:64] FLAG: --maximum-dead-containers="-1" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028348 2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028351 2569 flags.go:64] FLAG: --memory-manager-policy="None" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028354 2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028356 2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028359 2569 flags.go:64] FLAG: --node-ip="0.0.0.0" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028362 2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028370 2569 flags.go:64] FLAG: --node-status-max-images="50" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028373 2569 flags.go:64] FLAG: --node-status-update-frequency="10s" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028376 2569 flags.go:64] FLAG: --oom-score-adj="-999" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028386 2569 flags.go:64] FLAG: --pod-cidr="" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028389 2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:dc76bab72f320de3d4105c90d73c4fb139c09e20ce0fa8dcbc0cb59920d27dec" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028394 2569 flags.go:64] FLAG: --pod-manifest-path="" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028397 2569 flags.go:64] FLAG: --pod-max-pids="-1" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028400 2569 flags.go:64] FLAG: --pods-per-core="0" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028402 2569 flags.go:64] FLAG: --port="10250" Apr 16 08:38:57.031853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028405 2569 flags.go:64] FLAG: --protect-kernel-defaults="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028408 2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-077850e94ba3882e6" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028411 2569 flags.go:64] FLAG: --qos-reserved="" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028415 2569 flags.go:64] FLAG: --read-only-port="10255" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028418 2569 flags.go:64] FLAG: --register-node="true" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028421 2569 flags.go:64] FLAG: --register-schedulable="true" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028424 2569 flags.go:64] FLAG: --register-with-taints="" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028427 2569 flags.go:64] FLAG: --registry-burst="10" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028430 2569 flags.go:64] FLAG: --registry-qps="5" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028432 2569 flags.go:64] FLAG: --reserved-cpus="" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028435 2569 flags.go:64] FLAG: --reserved-memory="" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028439 2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028441 2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028444 2569 flags.go:64] FLAG: --rotate-certificates="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028447 2569 flags.go:64] FLAG: --rotate-server-certificates="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028450 2569 flags.go:64] FLAG: --runonce="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028453 2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028456 2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028459 2569 flags.go:64] FLAG: --seccomp-default="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028461 2569 flags.go:64] FLAG: --serialize-image-pulls="true" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028464 2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028467 2569 flags.go:64] FLAG: --storage-driver-db="cadvisor" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028470 2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028473 2569 flags.go:64] FLAG: --storage-driver-password="root" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028476 2569 flags.go:64] FLAG: --storage-driver-secure="false" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028478 2569 flags.go:64] FLAG: --storage-driver-table="stats" Apr 16 08:38:57.032446 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028481 2569 flags.go:64] FLAG: --storage-driver-user="root" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028490 2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028493 2569 flags.go:64] FLAG: --sync-frequency="1m0s" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028497 2569 flags.go:64] FLAG: --system-cgroups="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028499 2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028505 2569 flags.go:64] FLAG: --system-reserved-cgroup="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028507 2569 flags.go:64] FLAG: --tls-cert-file="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028510 2569 flags.go:64] FLAG: --tls-cipher-suites="[]" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028518 2569 flags.go:64] FLAG: --tls-min-version="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028521 2569 flags.go:64] FLAG: --tls-private-key-file="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028524 2569 flags.go:64] FLAG: --topology-manager-policy="none" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028527 2569 flags.go:64] FLAG: --topology-manager-policy-options="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028533 2569 flags.go:64] FLAG: --topology-manager-scope="container" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028536 2569 flags.go:64] FLAG: --v="2" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028540 2569 flags.go:64] FLAG: --version="false" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028544 2569 flags.go:64] FLAG: --vmodule="" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028548 2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.028551 2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028674 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028678 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028681 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028685 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028688 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028691 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:38:57.033071 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028693 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028695 2569 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028698 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028701 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028703 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028706 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028708 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028711 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028713 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028716 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028724 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028727 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028729 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028732 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028734 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028737 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028739 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028742 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028744 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028747 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:38:57.033685 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028751 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028753 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028756 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028758 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028761 2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028763 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028766 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028768 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028770 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028773 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028775 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028778 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028780 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028783 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028785 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028787 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028790 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028792 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028795 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:38:57.034206 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028797 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028801 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028804 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028808 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028817 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028822 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028825 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028827 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028830 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028832 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028835 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028837 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028840 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028844 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028847 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028849 2569 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028852 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028854 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028857 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:38:57.034688 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028859 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028861 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028864 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028866 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028869 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028871 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028874 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028876 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028879 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028881 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028884 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028886 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028888 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028891 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028893 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028896 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028898 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028901 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028909 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028911 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:38:57.035167 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028914 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:38:57.035681 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.028916 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:38:57.035681 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.029902 2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:38:57.036973 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.036951 2569 server.go:530] "Kubelet version" kubeletVersion="v1.33.9" Apr 16 08:38:57.037013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.036975 2569 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037023 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037029 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037032 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037035 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037038 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037040 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:38:57.037041 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037043 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037046 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037067 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037069 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037072 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037075 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037077 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037081 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037086 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037089 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037092 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037095 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037097 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037100 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037103 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037105 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037108 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037111 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037113 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:38:57.037237 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037116 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037118 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037121 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037123 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037126 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037128 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037132 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037135 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037137 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037140 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037142 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037145 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037148 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037152 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037155 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037160 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037163 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037165 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037168 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:38:57.037718 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037171 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037173 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037176 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037178 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037181 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037183 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037186 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037188 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037191 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037193 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037196 2569 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037198 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037201 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037203 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037206 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037208 2569 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037211 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037213 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037216 2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037218 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:38:57.038240 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037221 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037224 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037226 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037229 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037231 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037234 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037236 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037239 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037241 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037244 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037247 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037250 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037252 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037254 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037257 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037260 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037262 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037265 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037267 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037269 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:38:57.038716 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037272 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037274 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.037279 2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037386 2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037391 2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037394 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037396 2569 feature_gate.go:328] unrecognized feature gate: DualReplica Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037399 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037402 2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037404 2569 feature_gate.go:328] unrecognized feature gate: Example Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037407 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037409 2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037412 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037415 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037417 2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Apr 16 08:38:57.039218 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037420 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037423 2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037425 2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037427 2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037430 2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037432 2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037435 2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037438 2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037441 2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037443 2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037446 2569 feature_gate.go:328] unrecognized feature gate: NewOLM Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037448 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037450 2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037453 2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037457 2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037460 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037463 2569 feature_gate.go:328] unrecognized feature gate: OVNObservability Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037466 2569 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037469 2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Apr 16 08:38:57.039591 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037471 2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037473 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037476 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037478 2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037481 2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037483 2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037486 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037488 2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037491 2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037493 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037496 2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037498 2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037503 2569 feature_gate.go:328] unrecognized feature gate: Example2 Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037505 2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037508 2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037510 2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037513 2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037515 2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037517 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Apr 16 08:38:57.040116 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037520 2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037522 2569 feature_gate.go:328] unrecognized feature gate: SignatureStores Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037525 2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037527 2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037530 2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037532 2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037535 2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037537 2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037540 2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037542 2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037545 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037547 2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037550 2569 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037552 2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037555 2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037557 2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037560 2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037563 2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037567 2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037569 2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Apr 16 08:38:57.040550 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037572 2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037574 2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037576 2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037579 2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037581 2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037583 2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037586 2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037589 2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037591 2569 feature_gate.go:328] unrecognized feature gate: PinnedImages Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037594 2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037596 2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037599 2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037602 2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037604 2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037607 2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:57.037609 2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Apr 16 08:38:57.041019 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.037614 2569 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Apr 16 08:38:57.041437 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.038429 2569 server.go:962] "Client rotation is on, will bootstrap in background" Apr 16 08:38:57.041651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.041628 2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Apr 16 08:38:57.042681 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.042670 2569 server.go:1019] "Starting client certificate rotation" Apr 16 08:38:57.042806 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.042791 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 08:38:57.042844 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.042831 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Apr 16 08:38:57.070390 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.070368 2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 08:38:57.073072 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.073034 2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Apr 16 08:38:57.085488 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.085465 2569 log.go:25] "Validated CRI v1 runtime API" Apr 16 08:38:57.091790 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.091772 2569 log.go:25] "Validated CRI v1 image API" Apr 16 08:38:57.093291 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.093271 2569 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Apr 16 08:38:57.096991 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.096968 2569 fs.go:135] Filesystem UUIDs: map[0058521a-6e82-4e16-a2b3-84f49be135ef:/dev/nvme0n1p3 26b0aab7-5091-47e2-ad1f-b89e2f7f4cd8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Apr 16 08:38:57.097077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.096990 2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Apr 16 08:38:57.101794 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.101775 2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 08:38:57.102793 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.102673 2569 manager.go:217] Machine: {Timestamp:2026-04-16 08:38:57.100638535 +0000 UTC m=+0.462650455 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101355 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec289cd26857b95ca1091828224b9d27 SystemUUID:ec289cd2-6857-b95c-a109-1828224b9d27 BootID:75f0c5ac-f60c-4ae5-bf22-846125927187 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6098944 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1a:81:a9:4d:df Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1a:81:a9:4d:df Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9a:eb:e7:4c:b9:ee Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Apr 16 08:38:57.102793 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.102791 2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Apr 16 08:38:57.102929 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.102918 2569 manager.go:233] Version: {KernelVersion:5.14.0-570.104.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260401-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Apr 16 08:38:57.105881 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.105854 2569 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Apr 16 08:38:57.106034 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.105885 2569 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Apr 16 08:38:57.106092 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.106044 2569 topology_manager.go:138] "Creating topology manager with none policy" Apr 16 08:38:57.106092 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.106066 2569 container_manager_linux.go:306] "Creating device plugin manager" Apr 16 08:38:57.106092 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.106080 2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 08:38:57.106892 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.106880 2569 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Apr 16 08:38:57.108861 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.108850 2569 state_mem.go:36] "Initialized new in-memory state store" Apr 16 08:38:57.108980 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.108971 2569 server.go:1267] "Using root directory" path="/var/lib/kubelet" Apr 16 08:38:57.112072 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.112044 2569 kubelet.go:491] "Attempting to sync node with API server" Apr 16 08:38:57.112118 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.112090 2569 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Apr 16 08:38:57.112118 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.112110 2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Apr 16 08:38:57.112174 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.112125 2569 kubelet.go:397] "Adding apiserver pod source" Apr 16 08:38:57.112174 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.112138 2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Apr 16 08:38:57.113408 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.113396 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 08:38:57.113445 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.113416 2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Apr 16 08:38:57.116823 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.116808 2569 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.10-2.rhaos4.20.gita4d0894.el9" apiVersion="v1" Apr 16 08:38:57.119002 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.118983 2569 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Apr 16 08:38:57.120632 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120617 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Apr 16 08:38:57.120673 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120640 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Apr 16 08:38:57.120673 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120650 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Apr 16 08:38:57.120673 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120658 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Apr 16 08:38:57.120673 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120666 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Apr 16 08:38:57.120673 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120672 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120678 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120683 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120690 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120695 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120709 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Apr 16 08:38:57.120811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.120718 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Apr 16 08:38:57.121624 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.121615 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Apr 16 08:38:57.121624 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.121625 2569 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Apr 16 08:38:57.129356 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.129337 2569 watchdog_linux.go:99] "Systemd watchdog is not enabled" Apr 16 08:38:57.129429 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.129391 2569 server.go:1295] "Started kubelet" Apr 16 08:38:57.129568 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.129509 2569 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Apr 16 08:38:57.129668 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.129558 2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Apr 16 08:38:57.129668 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.129592 2569 server_v1.go:47] "podresources" method="list" useActivePods=true Apr 16 08:38:57.130379 ip-10-0-137-248 systemd[1]: Started Kubernetes Kubelet. Apr 16 08:38:57.131023 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.131009 2569 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Apr 16 08:38:57.132249 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.132226 2569 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Apr 16 08:38:57.132817 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.132803 2569 server.go:317] "Adding debug handlers to kubelet server" Apr 16 08:38:57.133123 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.133089 2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Apr 16 08:38:57.133244 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.133088 2569 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Apr 16 08:38:57.135290 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.134195 2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-248.ec2.internal.18a6c99334e9d5e7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-248.ec2.internal,UID:ip-10-0-137-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-248.ec2.internal,},FirstTimestamp:2026-04-16 08:38:57.129354727 +0000 UTC m=+0.491366647,LastTimestamp:2026-04-16 08:38:57.129354727 +0000 UTC m=+0.491366647,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-248.ec2.internal,}" Apr 16 08:38:57.140150 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.140130 2569 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Apr 16 08:38:57.140430 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.140414 2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Apr 16 08:38:57.141132 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141113 2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Apr 16 08:38:57.141603 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141582 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rrz4" Apr 16 08:38:57.141787 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141770 2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Apr 16 08:38:57.141787 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141789 2569 factory.go:55] Registering systemd factory Apr 16 08:38:57.141951 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141797 2569 factory.go:223] Registration of the systemd container factory successfully Apr 16 08:38:57.141951 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141836 2569 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Apr 16 08:38:57.141951 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141837 2569 volume_manager.go:295] "The desired_state_of_world populator starts" Apr 16 08:38:57.141951 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141861 2569 volume_manager.go:297] "Starting Kubelet Volume Manager" Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141966 2569 reconstruct.go:97] "Volume reconstruction finished" Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.141974 2569 reconciler.go:26] "Reconciler: start to sync state" Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.142018 2569 factory.go:153] Registering CRI-O factory Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.142033 2569 factory.go:223] Registration of the crio container factory successfully Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.142071 2569 factory.go:103] Registering Raw factory Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.142070 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.142130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.142087 2569 manager.go:1196] Started watching for new ooms in manager Apr 16 08:38:57.143009 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.142987 2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Apr 16 08:38:57.143290 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.143266 2569 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Apr 16 08:38:57.143358 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.143346 2569 manager.go:319] Starting recovery of all containers Apr 16 08:38:57.149067 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.148904 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5rrz4" Apr 16 08:38:57.152302 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.152280 2569 manager.go:324] Recovery completed Apr 16 08:38:57.156916 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.156898 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.159512 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159496 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.159576 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159525 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.159576 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159535 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.159946 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159932 2569 cpu_manager.go:222] "Starting CPU manager" policy="none" Apr 16 08:38:57.159946 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159944 2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Apr 16 08:38:57.160027 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.159958 2569 state_mem.go:36] "Initialized new in-memory state store" Apr 16 08:38:57.164034 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.164023 2569 policy_none.go:49] "None policy: Start" Apr 16 08:38:57.164096 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.164039 2569 memory_manager.go:186] "Starting memorymanager" policy="None" Apr 16 08:38:57.164096 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.164066 2569 state_mem.go:35] "Initializing new in-memory state store" Apr 16 08:38:57.207589 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.207573 2569 manager.go:341] "Starting Device Plugin manager" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.207602 2569 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.207613 2569 server.go:85] "Starting device plugin registration server" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.207840 2569 eviction_manager.go:189] "Eviction manager: starting control loop" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.207852 2569 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.207937 2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.208010 2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.208023 2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.208557 2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Apr 16 08:38:57.226974 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.208593 2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.250780 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.250751 2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Apr 16 08:38:57.252086 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.252072 2569 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Apr 16 08:38:57.252180 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.252095 2569 status_manager.go:230] "Starting to sync pod status with apiserver" Apr 16 08:38:57.252180 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.252116 2569 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Apr 16 08:38:57.252180 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.252125 2569 kubelet.go:2451] "Starting kubelet main sync loop" Apr 16 08:38:57.252180 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.252162 2569 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Apr 16 08:38:57.254823 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.254804 2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:38:57.308759 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.308684 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.309826 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.309810 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.309919 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.309843 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.309919 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.309853 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.309919 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.309875 2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.317570 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.317554 2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.317649 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.317577 2569 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-248.ec2.internal\": node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.339676 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.339653 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.352395 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.352365 2569 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal"] Apr 16 08:38:57.352469 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.352451 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.353233 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.353218 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.353295 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.353247 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.353295 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.353261 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.355732 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.355720 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.355891 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.355877 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.355924 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.355908 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.356419 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356403 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.356480 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356432 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.356480 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356442 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.356546 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356403 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.356546 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356511 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.356546 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.356523 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.359016 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.358998 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.359116 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.359032 2569 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Apr 16 08:38:57.359703 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.359690 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientMemory" Apr 16 08:38:57.359747 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.359715 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasNoDiskPressure" Apr 16 08:38:57.359747 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.359732 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeHasSufficientPID" Apr 16 08:38:57.382634 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.382615 2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-248.ec2.internal\" not found" node="ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.386975 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.386962 2569 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-248.ec2.internal\" not found" node="ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.439844 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.439814 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.443162 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.443146 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9f331420ac4d3ee0faecb39c870566-config\") pod \"kube-apiserver-proxy-ip-10-0-137-248.ec2.internal\" (UID: \"7a9f331420ac4d3ee0faecb39c870566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.443225 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.443172 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.443225 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.443192 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.540555 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.540523 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.543882 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543853 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9f331420ac4d3ee0faecb39c870566-config\") pod \"kube-apiserver-proxy-ip-10-0-137-248.ec2.internal\" (UID: \"7a9f331420ac4d3ee0faecb39c870566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.543948 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543892 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.543948 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543910 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.544028 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543952 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9f331420ac4d3ee0faecb39c870566-config\") pod \"kube-apiserver-proxy-ip-10-0-137-248.ec2.internal\" (UID: \"7a9f331420ac4d3ee0faecb39c870566\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.544028 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543974 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.544028 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.543978 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b655d58304bfe4a9ba45f3dc3916569-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal\" (UID: \"3b655d58304bfe4a9ba45f3dc3916569\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.641294 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.641236 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.685666 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.685627 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.689061 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:57.689033 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:57.741798 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.741764 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.842237 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.842211 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:57.942693 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:57.942672 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:58.042097 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.042066 2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Apr 16 08:38:58.042642 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.042221 2569 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Apr 16 08:38:58.043177 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:58.043161 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:58.113072 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.113034 2569 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:38:58.140886 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.140862 2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Apr 16 08:38:58.142367 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.142350 2569 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:38:58.143368 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:58.143345 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:58.148731 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.148714 2569 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Apr 16 08:38:58.152016 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.151994 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-04-15 08:33:57 +0000 UTC" deadline="2027-11-28 14:46:43.020440511 +0000 UTC" Apr 16 08:38:58.152016 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.152014 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14190h7m44.868428936s" Apr 16 08:38:58.172107 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.172087 2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4488v" Apr 16 08:38:58.177408 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.177391 2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4488v" Apr 16 08:38:58.243552 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:58.243489 2569 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-248.ec2.internal\" not found" Apr 16 08:38:58.280311 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:58.280265 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9f331420ac4d3ee0faecb39c870566.slice/crio-b7c2a0b90625d7711b2c8608cfa493493888ff604ee93c944942e1895f01a85e WatchSource:0}: Error finding container b7c2a0b90625d7711b2c8608cfa493493888ff604ee93c944942e1895f01a85e: Status 404 returned error can't find the container with id b7c2a0b90625d7711b2c8608cfa493493888ff604ee93c944942e1895f01a85e Apr 16 08:38:58.280502 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:58.280483 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b655d58304bfe4a9ba45f3dc3916569.slice/crio-244eb153e76881d86e2d1979f20136b0697d318f2005796ada9adae62fe8f255 WatchSource:0}: Error finding container 244eb153e76881d86e2d1979f20136b0697d318f2005796ada9adae62fe8f255: Status 404 returned error can't find the container with id 244eb153e76881d86e2d1979f20136b0697d318f2005796ada9adae62fe8f255 Apr 16 08:38:58.283970 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.283956 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 08:38:58.325300 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.325279 2569 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:38:58.341535 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.341518 2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" Apr 16 08:38:58.353443 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.353418 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 08:38:58.354477 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.354465 2569 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" Apr 16 08:38:58.361624 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:58.361608 2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Apr 16 08:38:59.113497 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.113443 2569 apiserver.go:52] "Watching apiserver" Apr 16 08:38:59.120253 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.120216 2569 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Apr 16 08:38:59.120635 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.120607 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-8zxqc","kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7","openshift-image-registry/node-ca-7gcs5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal","openshift-multus/multus-additional-cni-plugins-rzw5p","openshift-multus/multus-fjk5h","openshift-multus/network-metrics-daemon-mcq96","openshift-cluster-node-tuning-operator/tuned-lll26","openshift-dns/node-resolver-k2pss","openshift-network-diagnostics/network-check-target-jqclg","openshift-network-operator/iptables-alerter-cxp7b","openshift-ovn-kubernetes/ovnkube-node-xkwdc"] Apr 16 08:38:59.127583 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.127562 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.129605 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.129584 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.129720 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.129644 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.129840 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.129825 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Apr 16 08:38:59.129840 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.129833 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Apr 16 08:38:59.129982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.129965 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rpk6n\"" Apr 16 08:38:59.132859 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.132839 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.135314 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.135293 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.136504 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.136228 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Apr 16 08:38:59.136504 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.136257 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.136504 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.136416 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.136504 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.136443 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gjpcq\"" Apr 16 08:38:59.137008 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.136991 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Apr 16 08:38:59.137521 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.137506 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96fgd\"" Apr 16 08:38:59.137828 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.137813 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.138201 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.138183 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Apr 16 08:38:59.139509 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.139474 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-szt97\"" Apr 16 08:38:59.139509 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.139483 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Apr 16 08:38:59.139955 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.139772 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.139955 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.139776 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.140537 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.140171 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.141890 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.141872 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Apr 16 08:38:59.141977 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.141946 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-fgwx2\"" Apr 16 08:38:59.141977 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.141880 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Apr 16 08:38:59.142438 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.142419 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.142528 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.142497 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:38:59.142588 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.142510 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.146071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.146028 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.146713 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.146694 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.146955 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.146934 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t5cfx\"" Apr 16 08:38:59.149069 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.149034 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.150886 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.150866 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8gc9p\"" Apr 16 08:38:59.151112 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151097 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.151167 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151143 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.151214 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151190 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-multus-certs\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151263 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151231 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-etc-kubernetes\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151263 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151258 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-socket-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.151350 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151281 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ln52\" (UniqueName: \"kubernetes.io/projected/95de5e06-3d54-44cd-853a-e1a39cd48bca-kube-api-access-9ln52\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.151350 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151328 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-var-lib-kubelet\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.151447 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151359 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-etc-tuned\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.151447 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151393 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-k8s-cni-cncf-io\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151447 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151433 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151452 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-multus\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151496 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a824f8c-b47c-4092-8fa7-c2564991fd54-host\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.151498 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151538 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-os-release\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151569 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.151596 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151575 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-kubernetes\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151600 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-registration-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151626 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-etc-selinux\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151651 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-kubelet\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151700 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-conf-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151727 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g785\" (UniqueName: \"kubernetes.io/projected/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-kube-api-access-7g785\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151749 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a824f8c-b47c-4092-8fa7-c2564991fd54-serviceca\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151776 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-cnibin\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151799 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-tmp\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151822 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-bin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151846 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.151876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151872 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-konnectivity-ca\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151936 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-lib-modules\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151961 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-host\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.151987 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhjt\" (UniqueName: \"kubernetes.io/projected/62574210-d767-406c-b1d9-85faec81ed89-kube-api-access-qxhjt\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152028 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-os-release\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152074 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-socket-dir-parent\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152092 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7qt\" (UniqueName: \"kubernetes.io/projected/5a824f8c-b47c-4092-8fa7-c2564991fd54-kube-api-access-fz7qt\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152126 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152167 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-sys-fs\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152197 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-conf\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152220 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-systemd\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152243 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-netns\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152265 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-device-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152289 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-system-cni-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152327 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.152371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152367 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2b9f\" (UniqueName: \"kubernetes.io/projected/3759b22a-834b-4461-859a-8702206c5593-kube-api-access-d2b9f\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152401 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-agent-certs\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-run\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152457 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cnibin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152481 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cni-binary-copy\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-hostroot\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152549 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72btg\" (UniqueName: \"kubernetes.io/projected/1b29b84e-b6c8-46e1-891b-df2585f55350-kube-api-access-72btg\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152572 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-modprobe-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152593 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-system-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152614 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152649 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152682 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152709 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152734 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysconfig\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152755 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152778 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-sys\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.152959 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.152814 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-daemon-config\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.153651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.153328 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.153651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.153362 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Apr 16 08:38:59.153651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.153387 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zlvc6\"" Apr 16 08:38:59.153651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.153571 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.154023 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.154008 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.155747 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.155729 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Apr 16 08:38:59.156153 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156131 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zk9zs\"" Apr 16 08:38:59.156491 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156336 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Apr 16 08:38:59.156491 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156383 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Apr 16 08:38:59.156491 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156392 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Apr 16 08:38:59.156665 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156531 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Apr 16 08:38:59.156665 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.156582 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Apr 16 08:38:59.178395 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.178367 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 08:33:58 +0000 UTC" deadline="2028-02-01 19:02:15.743175394 +0000 UTC" Apr 16 08:38:59.178473 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.178397 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15754h23m16.564782248s" Apr 16 08:38:59.242945 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.242919 2569 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Apr 16 08:38:59.253511 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253484 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.253652 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253526 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysconfig\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.253652 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253553 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-etc-tuned\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.253652 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253581 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15236aa1-27f1-4a9e-8554-5260439b1f5d-iptables-alerter-script\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.253652 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.253615 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:38:59.253652 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253624 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-multus-certs\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253670 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-etc-kubernetes\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.253712 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:38:59.75367472 +0000 UTC m=+3.115686644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253711 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-multus-certs\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253710 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-etc-kubernetes\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253745 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysconfig\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253752 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-socket-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253785 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ln52\" (UniqueName: \"kubernetes.io/projected/95de5e06-3d54-44cd-853a-e1a39cd48bca-kube-api-access-9ln52\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253815 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-tmp-dir\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253839 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253863 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-socket-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.253895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253866 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sck6\" (UniqueName: \"kubernetes.io/projected/15236aa1-27f1-4a9e-8554-5260439b1f5d-kube-api-access-6sck6\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253906 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-k8s-cni-cncf-io\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253934 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-multus\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253959 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a824f8c-b47c-4092-8fa7-c2564991fd54-host\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253944 2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.253984 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-os-release\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254008 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-kubernetes\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254011 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-k8s-cni-cncf-io\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254025 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5a824f8c-b47c-4092-8fa7-c2564991fd54-host\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254019 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-multus\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254035 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-systemd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254094 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254106 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-kubernetes\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254110 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-os-release\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254126 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-registration-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254151 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-etc-selinux\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254177 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-kubelet\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.254404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254202 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7g785\" (UniqueName: \"kubernetes.io/projected/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-kube-api-access-7g785\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254205 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-registration-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254228 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhjt\" (UniqueName: \"kubernetes.io/projected/62574210-d767-406c-b1d9-85faec81ed89-kube-api-access-qxhjt\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254254 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15236aa1-27f1-4a9e-8554-5260439b1f5d-host-slash\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254258 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-kubelet\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254299 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-kubelet\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254307 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-etc-selinux\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254335 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-netd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254357 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-bin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254379 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254405 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7qt\" (UniqueName: \"kubernetes.io/projected/5a824f8c-b47c-4092-8fa7-c2564991fd54-kube-api-access-fz7qt\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254423 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-var-lib-cni-bin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254458 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254533 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-sys-fs\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254545 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254587 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-conf\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254593 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-sys-fs\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.255273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254615 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-systemd-units\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254637 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-bin\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254660 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-netns\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254693 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-device-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254706 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-conf\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254741 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/95de5e06-3d54-44cd-853a-e1a39cd48bca-device-dir\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254746 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-host-run-netns\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254767 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-agent-certs\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254794 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254823 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-node-log\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254847 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4x8p\" (UniqueName: \"kubernetes.io/projected/af3932be-73ab-4191-8a83-b4ead0398830-kube-api-access-c4x8p\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254876 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cnibin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254900 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-hostroot\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254926 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-72btg\" (UniqueName: \"kubernetes.io/projected/1b29b84e-b6c8-46e1-891b-df2585f55350-kube-api-access-72btg\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254940 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cnibin\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254952 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-ovn\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254942 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-hostroot\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254969 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.254984 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-config\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255099 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-system-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255127 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255148 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255169 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255171 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-system-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255183 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-sys\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255201 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-netns\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255205 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-cni-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255217 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255241 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-daemon-config\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255257 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-var-lib-kubelet\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255265 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-sysctl-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255271 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-hosts-file\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255288 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-var-lib-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255302 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-sys\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.256761 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255310 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-env-overrides\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255327 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-slash\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255344 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-conf-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255359 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a824f8c-b47c-4092-8fa7-c2564991fd54-serviceca\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255375 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-cnibin\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255396 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-tmp\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255410 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-konnectivity-ca\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255425 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-lib-modules\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255440 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-host\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255460 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-etc-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255482 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-os-release\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255497 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-socket-dir-parent\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255512 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-systemd\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255535 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-run\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255564 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5gdk\" (UniqueName: \"kubernetes.io/projected/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-kube-api-access-r5gdk\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255582 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-system-cni-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255600 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.257489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255629 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d2b9f\" (UniqueName: \"kubernetes.io/projected/3759b22a-834b-4461-859a-8702206c5593-kube-api-access-d2b9f\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255646 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cni-binary-copy\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255665 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-daemon-config\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255664 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-modprobe-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255704 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-log-socket\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255721 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af3932be-73ab-4191-8a83-b4ead0398830-ovn-node-metrics-cert\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255734 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-modprobe-d\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255739 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-script-lib\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255758 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255776 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-var-lib-kubelet\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255882 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-socket-dir-parent\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255885 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-os-release\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255913 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-multus-conf-dir\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255913 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-etc-systemd\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255952 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-host\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255958 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-run\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255991 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-system-cni-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256334 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5a824f8c-b47c-4092-8fa7-c2564991fd54-serviceca\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.258013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256412 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-binary-copy\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256427 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-konnectivity-ca\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.255919 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256493 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3759b22a-834b-4461-859a-8702206c5593-cnibin\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256536 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62574210-d767-406c-b1d9-85faec81ed89-lib-modules\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256569 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-cni-binary-copy\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.256760 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3759b22a-834b-4461-859a-8702206c5593-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.258561 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.257624 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-etc-tuned\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.258936 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.258912 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62574210-d767-406c-b1d9-85faec81ed89-tmp\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.259203 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.259180 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b10f80d7-ad09-4e3c-b75a-ec48f22e39be-agent-certs\") pod \"konnectivity-agent-8zxqc\" (UID: \"b10f80d7-ad09-4e3c-b75a-ec48f22e39be\") " pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.260281 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.260236 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" event={"ID":"7a9f331420ac4d3ee0faecb39c870566","Type":"ContainerStarted","Data":"b7c2a0b90625d7711b2c8608cfa493493888ff604ee93c944942e1895f01a85e"} Apr 16 08:38:59.261682 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.261659 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" event={"ID":"3b655d58304bfe4a9ba45f3dc3916569","Type":"ContainerStarted","Data":"244eb153e76881d86e2d1979f20136b0697d318f2005796ada9adae62fe8f255"} Apr 16 08:38:59.264224 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.264163 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ln52\" (UniqueName: \"kubernetes.io/projected/95de5e06-3d54-44cd-853a-e1a39cd48bca-kube-api-access-9ln52\") pod \"aws-ebs-csi-driver-node-hn6n7\" (UID: \"95de5e06-3d54-44cd-853a-e1a39cd48bca\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.265848 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.265794 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2b9f\" (UniqueName: \"kubernetes.io/projected/3759b22a-834b-4461-859a-8702206c5593-kube-api-access-d2b9f\") pod \"multus-additional-cni-plugins-rzw5p\" (UID: \"3759b22a-834b-4461-859a-8702206c5593\") " pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.266309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.266278 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7qt\" (UniqueName: \"kubernetes.io/projected/5a824f8c-b47c-4092-8fa7-c2564991fd54-kube-api-access-fz7qt\") pod \"node-ca-7gcs5\" (UID: \"5a824f8c-b47c-4092-8fa7-c2564991fd54\") " pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.266403 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.266387 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g785\" (UniqueName: \"kubernetes.io/projected/80259a0b-bf2a-4d3c-886c-62ed1c5b20b6-kube-api-access-7g785\") pod \"multus-fjk5h\" (UID: \"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6\") " pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.267007 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.266987 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72btg\" (UniqueName: \"kubernetes.io/projected/1b29b84e-b6c8-46e1-891b-df2585f55350-kube-api-access-72btg\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.267425 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.267408 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhjt\" (UniqueName: \"kubernetes.io/projected/62574210-d767-406c-b1d9-85faec81ed89-kube-api-access-qxhjt\") pod \"tuned-lll26\" (UID: \"62574210-d767-406c-b1d9-85faec81ed89\") " pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.356800 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356759 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-systemd-units\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.356800 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356805 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-bin\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356836 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356862 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-node-log\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356884 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c4x8p\" (UniqueName: \"kubernetes.io/projected/af3932be-73ab-4191-8a83-b4ead0398830-kube-api-access-c4x8p\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356890 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-systemd-units\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356922 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-node-log\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356901 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-ovn\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356956 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-ovn\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356961 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356895 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-bin\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.356970 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-config\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357086 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-netns\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357113 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357138 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-hosts-file\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357160 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-var-lib-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357182 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-env-overrides\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357205 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-slash\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357232 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-netns\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357238 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-etc-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357269 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r5gdk\" (UniqueName: \"kubernetes.io/projected/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-kube-api-access-r5gdk\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.357332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357299 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-log-socket\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357347 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af3932be-73ab-4191-8a83-b4ead0398830-ovn-node-metrics-cert\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357373 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-script-lib\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357415 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15236aa1-27f1-4a9e-8554-5260439b1f5d-iptables-alerter-script\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357446 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-tmp-dir\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357474 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357503 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6sck6\" (UniqueName: \"kubernetes.io/projected/15236aa1-27f1-4a9e-8554-5260439b1f5d-kube-api-access-6sck6\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357581 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-systemd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357608 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-env-overrides\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357628 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357610 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357875 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357662 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-var-lib-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357875 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357661 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-config\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.357875 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357741 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-hosts-file\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.357875 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.357206 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-run-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358100 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-etc-openvswitch\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358137 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-log-socket\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358144 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-run-systemd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358154 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358175 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15236aa1-27f1-4a9e-8554-5260439b1f5d-host-slash\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358174 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af3932be-73ab-4191-8a83-b4ead0398830-ovnkube-script-lib\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358200 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/15236aa1-27f1-4a9e-8554-5260439b1f5d-host-slash\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358221 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-slash\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358234 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-kubelet\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358283 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-netd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358311 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-kubelet\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.358402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358367 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af3932be-73ab-4191-8a83-b4ead0398830-host-cni-netd\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.359030 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358401 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-tmp-dir\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.359030 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.358485 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/15236aa1-27f1-4a9e-8554-5260439b1f5d-iptables-alerter-script\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.361197 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.361174 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af3932be-73ab-4191-8a83-b4ead0398830-ovn-node-metrics-cert\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.362950 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.362927 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:38:59.362950 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.362953 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:38:59.363145 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.362966 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:38:59.363145 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.363024 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:38:59.863007399 +0000 UTC m=+3.225019347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:38:59.364866 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.364818 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4x8p\" (UniqueName: \"kubernetes.io/projected/af3932be-73ab-4191-8a83-b4ead0398830-kube-api-access-c4x8p\") pod \"ovnkube-node-xkwdc\" (UID: \"af3932be-73ab-4191-8a83-b4ead0398830\") " pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.365430 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.365408 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5gdk\" (UniqueName: \"kubernetes.io/projected/0ee13c74-b0d0-42a3-959b-963a8f04c0b0-kube-api-access-r5gdk\") pod \"node-resolver-k2pss\" (UID: \"0ee13c74-b0d0-42a3-959b-963a8f04c0b0\") " pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.365731 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.365711 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sck6\" (UniqueName: \"kubernetes.io/projected/15236aa1-27f1-4a9e-8554-5260439b1f5d-kube-api-access-6sck6\") pod \"iptables-alerter-cxp7b\" (UID: \"15236aa1-27f1-4a9e-8554-5260439b1f5d\") " pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.440082 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.440035 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fjk5h" Apr 16 08:38:59.447991 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.447959 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" Apr 16 08:38:59.458638 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.458616 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" Apr 16 08:38:59.462776 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.462761 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7gcs5" Apr 16 08:38:59.470301 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.470280 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:38:59.475806 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.475789 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lll26" Apr 16 08:38:59.482366 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.482347 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k2pss" Apr 16 08:38:59.489850 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.489831 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxp7b" Apr 16 08:38:59.495479 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.495462 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:38:59.509784 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.509762 2569 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Apr 16 08:38:59.760689 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.760658 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:38:59.760849 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.760795 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:38:59.760896 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.760851 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:00.76083846 +0000 UTC m=+4.122850369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:38:59.928517 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.928488 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a824f8c_b47c_4092_8fa7_c2564991fd54.slice/crio-c814b72a1cb5365b7d4ce154a718e89aa81e7b8146658a14c519bbb6cb1eb895 WatchSource:0}: Error finding container c814b72a1cb5365b7d4ce154a718e89aa81e7b8146658a14c519bbb6cb1eb895: Status 404 returned error can't find the container with id c814b72a1cb5365b7d4ce154a718e89aa81e7b8146658a14c519bbb6cb1eb895 Apr 16 08:38:59.929087 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.929043 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62574210_d767_406c_b1d9_85faec81ed89.slice/crio-f19fccc64eff49a46247a42ac9f17f6369e2ec49f5a234e853c71556ccd52768 WatchSource:0}: Error finding container f19fccc64eff49a46247a42ac9f17f6369e2ec49f5a234e853c71556ccd52768: Status 404 returned error can't find the container with id f19fccc64eff49a46247a42ac9f17f6369e2ec49f5a234e853c71556ccd52768 Apr 16 08:38:59.930290 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.930266 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15236aa1_27f1_4a9e_8554_5260439b1f5d.slice/crio-7b004f722203ea8f5f4ae11379ccb6a118b8a2252bb452e73680a424dbf2e48b WatchSource:0}: Error finding container 7b004f722203ea8f5f4ae11379ccb6a118b8a2252bb452e73680a424dbf2e48b: Status 404 returned error can't find the container with id 7b004f722203ea8f5f4ae11379ccb6a118b8a2252bb452e73680a424dbf2e48b Apr 16 08:38:59.931042 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.931021 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10f80d7_ad09_4e3c_b75a_ec48f22e39be.slice/crio-47b11620e664fead94433fc2a17b918ce553f937403a0aab7ee3a7ca1b611a80 WatchSource:0}: Error finding container 47b11620e664fead94433fc2a17b918ce553f937403a0aab7ee3a7ca1b611a80: Status 404 returned error can't find the container with id 47b11620e664fead94433fc2a17b918ce553f937403a0aab7ee3a7ca1b611a80 Apr 16 08:38:59.931866 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.931850 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3759b22a_834b_4461_859a_8702206c5593.slice/crio-4e2e62e7794d6ea600a9fb91b7ac629a508d9108960686a3ae45ebcef9f88567 WatchSource:0}: Error finding container 4e2e62e7794d6ea600a9fb91b7ac629a508d9108960686a3ae45ebcef9f88567: Status 404 returned error can't find the container with id 4e2e62e7794d6ea600a9fb91b7ac629a508d9108960686a3ae45ebcef9f88567 Apr 16 08:38:59.932739 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:38:59.932627 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80259a0b_bf2a_4d3c_886c_62ed1c5b20b6.slice/crio-4688c791a4a13de6d51d45934ee571178776569cbf7b86ddc49f247a4e287cf6 WatchSource:0}: Error finding container 4688c791a4a13de6d51d45934ee571178776569cbf7b86ddc49f247a4e287cf6: Status 404 returned error can't find the container with id 4688c791a4a13de6d51d45934ee571178776569cbf7b86ddc49f247a4e287cf6 Apr 16 08:38:59.961758 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:38:59.961735 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:38:59.961872 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.961857 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:38:59.961937 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.961879 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:38:59.961937 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.961890 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:38:59.962009 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:38:59.961946 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:00.96192652 +0000 UTC m=+4.323938442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:00.179583 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.179405 2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-04-15 08:33:58 +0000 UTC" deadline="2027-12-18 19:08:40.640083359 +0000 UTC" Apr 16 08:39:00.179583 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.179574 2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14674h29m40.46051199s" Apr 16 08:39:00.253032 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.253000 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:00.253210 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.253162 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:00.265323 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.265229 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lll26" event={"ID":"62574210-d767-406c-b1d9-85faec81ed89","Type":"ContainerStarted","Data":"f19fccc64eff49a46247a42ac9f17f6369e2ec49f5a234e853c71556ccd52768"} Apr 16 08:39:00.266635 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.266609 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxp7b" event={"ID":"15236aa1-27f1-4a9e-8554-5260439b1f5d","Type":"ContainerStarted","Data":"7b004f722203ea8f5f4ae11379ccb6a118b8a2252bb452e73680a424dbf2e48b"} Apr 16 08:39:00.267800 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.267750 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" event={"ID":"95de5e06-3d54-44cd-853a-e1a39cd48bca","Type":"ContainerStarted","Data":"4f3ae2213a25c7b4a1e44ab4499e7a5f1551a050d12c4e7271136e36b2a3d926"} Apr 16 08:39:00.269612 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.269585 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zxqc" event={"ID":"b10f80d7-ad09-4e3c-b75a-ec48f22e39be","Type":"ContainerStarted","Data":"47b11620e664fead94433fc2a17b918ce553f937403a0aab7ee3a7ca1b611a80"} Apr 16 08:39:00.275611 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.275023 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7gcs5" event={"ID":"5a824f8c-b47c-4092-8fa7-c2564991fd54","Type":"ContainerStarted","Data":"c814b72a1cb5365b7d4ce154a718e89aa81e7b8146658a14c519bbb6cb1eb895"} Apr 16 08:39:00.278183 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.278157 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" event={"ID":"7a9f331420ac4d3ee0faecb39c870566","Type":"ContainerStarted","Data":"84a69ebb95cf2dc36307137293ca4936fef4bbd1bd1253693a0e284d20c2115b"} Apr 16 08:39:00.279863 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.279829 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"e8f6dbe1715fbcad5f3055f4b2ecb9508fc40074419ff22a59cc94ea3e64e0ff"} Apr 16 08:39:00.281903 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.281856 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k2pss" event={"ID":"0ee13c74-b0d0-42a3-959b-963a8f04c0b0","Type":"ContainerStarted","Data":"936dabe8f3d93df685fb61fdd7c9402bbcc9bf5c55ee3cffbc3e49312b7e8ea7"} Apr 16 08:39:00.287210 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.287185 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjk5h" event={"ID":"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6","Type":"ContainerStarted","Data":"4688c791a4a13de6d51d45934ee571178776569cbf7b86ddc49f247a4e287cf6"} Apr 16 08:39:00.289152 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.289133 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerStarted","Data":"4e2e62e7794d6ea600a9fb91b7ac629a508d9108960686a3ae45ebcef9f88567"} Apr 16 08:39:00.289568 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.289525 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-248.ec2.internal" podStartSLOduration=2.289514031 podStartE2EDuration="2.289514031s" podCreationTimestamp="2026-04-16 08:38:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:39:00.289420045 +0000 UTC m=+3.651431976" watchObservedRunningTime="2026-04-16 08:39:00.289514031 +0000 UTC m=+3.651525964" Apr 16 08:39:00.769040 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.769009 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:00.769241 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.769178 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:00.769241 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.769230 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:02.769216953 +0000 UTC m=+6.131228861 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:00.970908 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:00.970871 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:00.971109 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.971091 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:00.971184 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.971119 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:00.971184 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.971132 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:00.971285 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:00.971188 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:02.971170546 +0000 UTC m=+6.333182470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:01.255071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:01.254996 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:01.255572 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:01.255141 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:02.252410 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:02.252369 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:02.252599 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.252524 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:02.300604 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:02.300546 2569 generic.go:358] "Generic (PLEG): container finished" podID="3b655d58304bfe4a9ba45f3dc3916569" containerID="fac99ded85d46abdfb84d2f4f253f2c4d85df68b9a3f14bae7009b29622292ab" exitCode=0 Apr 16 08:39:02.300604 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:02.300596 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" event={"ID":"3b655d58304bfe4a9ba45f3dc3916569","Type":"ContainerDied","Data":"fac99ded85d46abdfb84d2f4f253f2c4d85df68b9a3f14bae7009b29622292ab"} Apr 16 08:39:02.787798 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:02.787761 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:02.787965 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.787955 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:02.788031 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.788020 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:06.788001554 +0000 UTC m=+10.150013477 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:02.990473 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:02.989853 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:02.990473 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.990029 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:02.990473 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.990066 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:02.990473 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.990079 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:02.990473 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:02.990139 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:06.990121158 +0000 UTC m=+10.352133072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:03.253465 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:03.252972 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:03.253465 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:03.253113 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:04.253262 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:04.253220 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:04.253703 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:04.253356 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:05.254603 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:05.254137 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:05.254603 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:05.254250 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:06.252733 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:06.252702 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:06.252908 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:06.252838 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:06.821310 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:06.820667 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:06.821310 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:06.820860 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:06.821310 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:06.820926 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:14.820905891 +0000 UTC m=+18.182917800 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:07.022652 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:07.022571 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:07.022850 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:07.022764 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:07.022850 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:07.022783 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:07.022850 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:07.022795 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:07.023025 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:07.022852 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:15.02283465 +0000 UTC m=+18.384846563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:07.253208 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:07.253179 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:07.253447 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:07.253272 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:08.253282 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.253250 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:08.253731 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:08.253379 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:08.887787 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.887754 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-4vp8m"] Apr 16 08:39:08.895555 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.895531 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:08.895691 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:08.895657 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:08.939140 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.939104 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-kubelet-config\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:08.939293 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.939153 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-dbus\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:08.939353 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:08.939278 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040467 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.040423 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-kubelet-config\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040633 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.040502 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-dbus\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040633 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.040554 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-kubelet-config\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040633 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.040574 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040791 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:09.040668 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:09.040791 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.040708 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1597cea1-f09d-46f2-9513-72dde7873326-dbus\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.040791 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:09.040720 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:09.540702692 +0000 UTC m=+12.902714603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:09.252963 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.252926 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:09.253148 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:09.253045 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:09.545015 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:09.544919 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:09.545429 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:09.545018 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:09.545429 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:09.545102 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:10.545084314 +0000 UTC m=+13.907096227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:10.253341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:10.253301 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:10.253517 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:10.253301 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:10.253517 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:10.253420 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:10.253517 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:10.253490 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:10.552586 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:10.552509 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:10.553000 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:10.552663 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:10.553000 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:10.552730 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:12.552711815 +0000 UTC m=+15.914723730 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:11.252732 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:11.252700 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:11.252903 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:11.252823 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:12.252760 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:12.252722 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:12.253231 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:12.252737 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:12.253231 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:12.252846 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:12.253231 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:12.252946 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:12.567238 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:12.567217 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:12.567336 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:12.567324 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:12.567391 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:12.567381 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:16.567367144 +0000 UTC m=+19.929379052 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:13.253313 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:13.253284 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:13.253829 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:13.253397 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:14.252595 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:14.252514 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:14.252765 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:14.252515 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:14.252765 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:14.252732 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:14.252964 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:14.252941 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:14.883765 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:14.883723 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:14.884288 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:14.884267 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:14.884346 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:14.884337 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:30.884317612 +0000 UTC m=+34.246329523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:15.085934 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:15.085896 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:15.086138 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:15.086098 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:15.086138 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:15.086124 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:15.086227 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:15.086139 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:15.086227 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:15.086195 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:31.086179609 +0000 UTC m=+34.448191517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:15.252863 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:15.252828 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:15.253083 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:15.252951 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:16.253189 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:16.253156 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:16.253611 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:16.253278 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:16.253703 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:16.253692 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:16.253801 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:16.253777 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:16.596744 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:16.596661 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:16.596906 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:16.596830 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:16.596963 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:16.596909 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:24.596890309 +0000 UTC m=+27.958902232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:17.253979 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:17.253952 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:17.254400 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:17.254203 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:17.347418 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:17.346770 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" event={"ID":"3b655d58304bfe4a9ba45f3dc3916569","Type":"ContainerStarted","Data":"107f3865add40b85a0a79672b3d94b133fca78713cdf3453dcd30cea3b9c53e9"} Apr 16 08:39:17.365120 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:17.364771 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-248.ec2.internal" podStartSLOduration=19.364751814 podStartE2EDuration="19.364751814s" podCreationTimestamp="2026-04-16 08:38:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:39:17.358291051 +0000 UTC m=+20.720302976" watchObservedRunningTime="2026-04-16 08:39:17.364751814 +0000 UTC m=+20.726763745" Apr 16 08:39:18.253070 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.253017 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:18.253240 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:18.253172 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:18.253538 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.253017 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:18.253685 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:18.253646 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:18.349709 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.349670 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" event={"ID":"95de5e06-3d54-44cd-853a-e1a39cd48bca","Type":"ContainerStarted","Data":"5c4304d68717743f8c639e8749ce6d22e16efe285a62f258603734644e47cbf6"} Apr 16 08:39:18.351174 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.351140 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8zxqc" event={"ID":"b10f80d7-ad09-4e3c-b75a-ec48f22e39be","Type":"ContainerStarted","Data":"618099159f6bbda2b6df905f25d7d6181cc0f9ec936cc1be64fc3a3577dd8c9a"} Apr 16 08:39:18.352602 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.352581 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7gcs5" event={"ID":"5a824f8c-b47c-4092-8fa7-c2564991fd54","Type":"ContainerStarted","Data":"113fca5838a9676da807a0ae17562106f201c394c043f43be3c9d7d463d29135"} Apr 16 08:39:18.354982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.354961 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:39:18.355267 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355251 2569 generic.go:358] "Generic (PLEG): container finished" podID="af3932be-73ab-4191-8a83-b4ead0398830" containerID="701825e2dc7319d3decb6118533c4193fae8ea49926f1a4c26abcdfa34daa697" exitCode=1 Apr 16 08:39:18.355340 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355310 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"6b18061ec21bf3eefe62934b38fba433019cba231231274eb7aa97f51d06f543"} Apr 16 08:39:18.355340 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355331 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"681311b9b0e67a32bd236fd07335adf69bb3f1d777f443e23ef12402e3f70a60"} Apr 16 08:39:18.355412 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355342 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"88756ec565d9d9be73b21bf93296c0021ce1da36af5600919adbe6a53d1da8cf"} Apr 16 08:39:18.355412 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355352 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"b8c0927524d88a11da5262bf371e7d2f1e241327b962fb20c771bb598f21e863"} Apr 16 08:39:18.355412 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355360 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerDied","Data":"701825e2dc7319d3decb6118533c4193fae8ea49926f1a4c26abcdfa34daa697"} Apr 16 08:39:18.355412 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.355368 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"0769d9aa4d03c69aca0b7c97e1d3382e60d5986848711adf7826210c241de71e"} Apr 16 08:39:18.356415 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.356397 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k2pss" event={"ID":"0ee13c74-b0d0-42a3-959b-963a8f04c0b0","Type":"ContainerStarted","Data":"3150fe4e073e6f13d6a1c197cba9d73fb8f06e4a45964f6158311e6440cd2b76"} Apr 16 08:39:18.357558 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.357533 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fjk5h" event={"ID":"80259a0b-bf2a-4d3c-886c-62ed1c5b20b6","Type":"ContainerStarted","Data":"b2c0b3579975ac19b6b291f1fe747a1052271e3e5e3165f6941dcfb7bca5f608"} Apr 16 08:39:18.358726 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.358708 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="88867bcc36707e76a0f2a2b4b5e270a7807604616259a112f8ce107bb3d89120" exitCode=0 Apr 16 08:39:18.358809 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.358759 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"88867bcc36707e76a0f2a2b4b5e270a7807604616259a112f8ce107bb3d89120"} Apr 16 08:39:18.359977 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.359939 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lll26" event={"ID":"62574210-d767-406c-b1d9-85faec81ed89","Type":"ContainerStarted","Data":"18acd1f878e6cc33e3d45861babb4b0af7e50131958d42bff119a4dd0c9c3f70"} Apr 16 08:39:18.362093 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.362036 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8zxqc" podStartSLOduration=4.188079012 podStartE2EDuration="21.362025738s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.932941253 +0000 UTC m=+3.294953161" lastFinishedPulling="2026-04-16 08:39:17.106887977 +0000 UTC m=+20.468899887" observedRunningTime="2026-04-16 08:39:18.361588694 +0000 UTC m=+21.723600626" watchObservedRunningTime="2026-04-16 08:39:18.362025738 +0000 UTC m=+21.724037667" Apr 16 08:39:18.370822 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.370782 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7gcs5" podStartSLOduration=4.150400114 podStartE2EDuration="21.370772428s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.931834595 +0000 UTC m=+3.293846517" lastFinishedPulling="2026-04-16 08:39:17.15220692 +0000 UTC m=+20.514218831" observedRunningTime="2026-04-16 08:39:18.370759745 +0000 UTC m=+21.732771674" watchObservedRunningTime="2026-04-16 08:39:18.370772428 +0000 UTC m=+21.732784357" Apr 16 08:39:18.380653 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.380619 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-k2pss" podStartSLOduration=4.232843306 podStartE2EDuration="21.380609885s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.959159596 +0000 UTC m=+3.321171503" lastFinishedPulling="2026-04-16 08:39:17.106926174 +0000 UTC m=+20.468938082" observedRunningTime="2026-04-16 08:39:18.380556609 +0000 UTC m=+21.742568539" watchObservedRunningTime="2026-04-16 08:39:18.380609885 +0000 UTC m=+21.742621815" Apr 16 08:39:18.395725 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.394551 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-fjk5h" podStartSLOduration=4.18156813 podStartE2EDuration="21.394530901s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.959253153 +0000 UTC m=+3.321265063" lastFinishedPulling="2026-04-16 08:39:17.172215909 +0000 UTC m=+20.534227834" observedRunningTime="2026-04-16 08:39:18.393989014 +0000 UTC m=+21.756000945" watchObservedRunningTime="2026-04-16 08:39:18.394530901 +0000 UTC m=+21.756542831" Apr 16 08:39:18.428283 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.428244 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lll26" podStartSLOduration=4.228190928 podStartE2EDuration="21.428234911s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.93114819 +0000 UTC m=+3.293160105" lastFinishedPulling="2026-04-16 08:39:17.131192177 +0000 UTC m=+20.493204088" observedRunningTime="2026-04-16 08:39:18.427809328 +0000 UTC m=+21.789821257" watchObservedRunningTime="2026-04-16 08:39:18.428234911 +0000 UTC m=+21.790246840" Apr 16 08:39:18.738840 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:18.738797 2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Apr 16 08:39:19.218105 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.217965 2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-04-16T08:39:18.738833886Z","UUID":"1d9fd466-c21a-4571-81e7-1aa6e7ef7ecb","Handler":null,"Name":"","Endpoint":""} Apr 16 08:39:19.220982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.220949 2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Apr 16 08:39:19.220982 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.220980 2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Apr 16 08:39:19.253036 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.253009 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:19.253207 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:19.253150 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:19.363827 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.363796 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxp7b" event={"ID":"15236aa1-27f1-4a9e-8554-5260439b1f5d","Type":"ContainerStarted","Data":"ad391d60eb28f235c5ab5a179fa8eb9f5b5b4689627fb769765d22b519ddce85"} Apr 16 08:39:19.365773 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.365674 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" event={"ID":"95de5e06-3d54-44cd-853a-e1a39cd48bca","Type":"ContainerStarted","Data":"c3550444d8237570b259eae815c10139473c81ac2258499675fccf3b7816df53"} Apr 16 08:39:19.381689 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:19.381638 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cxp7b" podStartSLOduration=5.182755027 podStartE2EDuration="22.381610255s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.932410219 +0000 UTC m=+3.294422144" lastFinishedPulling="2026-04-16 08:39:17.131265451 +0000 UTC m=+20.493277372" observedRunningTime="2026-04-16 08:39:19.381166222 +0000 UTC m=+22.743178152" watchObservedRunningTime="2026-04-16 08:39:19.381610255 +0000 UTC m=+22.743622188" Apr 16 08:39:20.253020 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:20.252988 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:20.253157 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:20.253135 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:20.253436 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:20.252994 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:20.253528 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:20.253493 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:20.378188 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:20.378160 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:39:20.378689 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:20.378659 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"9bfd8b6d97c543f1bcec4241cb747a24dc7ea8217160f3817f53e3907fce3f87"} Apr 16 08:39:21.253097 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:21.253045 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:21.253280 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:21.253190 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:21.383186 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:21.383147 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" event={"ID":"95de5e06-3d54-44cd-853a-e1a39cd48bca","Type":"ContainerStarted","Data":"8447396641b65afdea505af9f12fb1b6673b8cfd4b21f534e48b4a2aeb8daa94"} Apr 16 08:39:21.398850 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:21.398803 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hn6n7" podStartSLOduration=4.152178917 podStartE2EDuration="24.398786015s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.959221768 +0000 UTC m=+3.321233677" lastFinishedPulling="2026-04-16 08:39:20.205828854 +0000 UTC m=+23.567840775" observedRunningTime="2026-04-16 08:39:21.398270927 +0000 UTC m=+24.760282868" watchObservedRunningTime="2026-04-16 08:39:21.398786015 +0000 UTC m=+24.760797942" Apr 16 08:39:22.252839 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:22.252809 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:22.253008 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:22.252809 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:22.253008 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:22.252917 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:22.253120 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:22.253013 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:22.643785 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:22.643752 2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:39:22.645011 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:22.644993 2569 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:39:23.252482 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.252451 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:23.252648 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:23.252557 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:23.388120 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.387906 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="f64acc440e824ae2a82e56d737d9c0e72121b1a7a9abe6a6d37b010b70a42dd0" exitCode=0 Apr 16 08:39:23.388267 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.387993 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"f64acc440e824ae2a82e56d737d9c0e72121b1a7a9abe6a6d37b010b70a42dd0"} Apr 16 08:39:23.391272 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.391256 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:39:23.391618 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.391587 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"3e8b5b87aa00894137d2afefedd34e8202c286a7707a50119d358a11e83817ad"} Apr 16 08:39:23.391827 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.391793 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:39:23.392161 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.392144 2569 scope.go:117] "RemoveContainer" containerID="701825e2dc7319d3decb6118533c4193fae8ea49926f1a4c26abcdfa34daa697" Apr 16 08:39:23.392246 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:23.392214 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8zxqc" Apr 16 08:39:24.252477 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.252453 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:24.252891 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.252454 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:24.252891 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.252577 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:24.252891 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.252618 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:24.396165 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.396090 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:39:24.396484 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.396450 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" event={"ID":"af3932be-73ab-4191-8a83-b4ead0398830","Type":"ContainerStarted","Data":"b6cfbaec5e1da9b111d4e8becbc31db8c24eecf4d376c34d7982cb41c3471ae9"} Apr 16 08:39:24.396779 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.396757 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:39:24.396891 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.396806 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:39:24.396891 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.396820 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:39:24.398916 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.398897 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="5d64aaeb8ad7170f7c43d44d9f3de734fab9b2410bfd73a6248cdb8941901b25" exitCode=0 Apr 16 08:39:24.399013 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.398958 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"5d64aaeb8ad7170f7c43d44d9f3de734fab9b2410bfd73a6248cdb8941901b25"} Apr 16 08:39:24.414018 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.413992 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:39:24.415560 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.415537 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:39:24.418507 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.418477 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vp8m"] Apr 16 08:39:24.418612 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.418565 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:24.418697 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.418676 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:24.422191 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.422164 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqclg"] Apr 16 08:39:24.422292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.422280 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:24.422372 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.422356 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:24.422790 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.422773 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mcq96"] Apr 16 08:39:24.422847 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.422841 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:24.422928 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.422914 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:24.426926 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.426888 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" podStartSLOduration=10.178753108 podStartE2EDuration="27.426877021s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.959187878 +0000 UTC m=+3.321199801" lastFinishedPulling="2026-04-16 08:39:17.207311792 +0000 UTC m=+20.569323714" observedRunningTime="2026-04-16 08:39:24.426510243 +0000 UTC m=+27.788522390" watchObservedRunningTime="2026-04-16 08:39:24.426877021 +0000 UTC m=+27.788888950" Apr 16 08:39:24.653935 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:24.653906 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:24.654101 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.654071 2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:24.654149 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:24.654139 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret podName:1597cea1-f09d-46f2-9513-72dde7873326 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:40.654119838 +0000 UTC m=+44.016131748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret") pod "global-pull-secret-syncer-4vp8m" (UID: "1597cea1-f09d-46f2-9513-72dde7873326") : object "kube-system"/"original-pull-secret" not registered Apr 16 08:39:25.403240 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:25.403204 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="98f0f8230201a9c37f6d4b1a75a76b468d712aa17c9c09aa3d656a6ec5b511d0" exitCode=0 Apr 16 08:39:25.403683 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:25.403289 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"98f0f8230201a9c37f6d4b1a75a76b468d712aa17c9c09aa3d656a6ec5b511d0"} Apr 16 08:39:26.252373 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:26.252334 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:26.252590 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:26.252334 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:26.252590 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:26.252470 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:26.252590 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:26.252508 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:26.252590 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:26.252333 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:26.252825 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:26.252636 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:28.252825 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:28.252788 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:28.252825 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:28.252820 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:28.253352 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:28.252800 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:28.253352 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:28.252907 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:28.253352 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:28.253008 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:28.253352 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:28.253118 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:30.252532 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.252262 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:30.253043 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.252260 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:30.253043 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.252640 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:39:30.253043 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.252719 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4vp8m" podUID="1597cea1-f09d-46f2-9513-72dde7873326" Apr 16 08:39:30.253043 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.252260 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:30.253043 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.252808 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jqclg" podUID="7315ebc3-7fa9-49f2-be69-7b9d6f0e5934" Apr 16 08:39:30.448461 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.448436 2569 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-248.ec2.internal" event="NodeReady" Apr 16 08:39:30.448613 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.448562 2569 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Apr 16 08:39:30.478943 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.478908 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:39:30.483371 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.483350 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg"] Apr 16 08:39:30.483533 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.483515 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.485893 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.485866 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Apr 16 08:39:30.486108 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.486090 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v9nxr\"" Apr 16 08:39:30.486220 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.485871 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Apr 16 08:39:30.486475 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.486458 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Apr 16 08:39:30.487899 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.487526 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h"] Apr 16 08:39:30.487899 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.487820 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.491278 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.491242 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Apr 16 08:39:30.491499 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.491469 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Apr 16 08:39:30.491597 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.491583 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Apr 16 08:39:30.491983 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.491964 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Apr 16 08:39:30.492325 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.492256 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx"] Apr 16 08:39:30.492590 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.492567 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.494566 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.494220 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Apr 16 08:39:30.494566 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.494413 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Apr 16 08:39:30.494712 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.494639 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Apr 16 08:39:30.495179 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.494921 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Apr 16 08:39:30.497402 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.496889 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Apr 16 08:39:30.499130 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.499110 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:39:30.499660 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.499137 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx"] Apr 16 08:39:30.499660 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.499150 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h"] Apr 16 08:39:30.499660 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.499165 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-h7fxz"] Apr 16 08:39:30.499660 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.499240 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.502219 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.502200 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-htctl\"" Apr 16 08:39:30.502452 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.502436 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Apr 16 08:39:30.503027 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.502986 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.505714 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.504853 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Apr 16 08:39:30.505714 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.505106 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Apr 16 08:39:30.506009 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.505989 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9d2hv\"" Apr 16 08:39:30.506135 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.506118 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg"] Apr 16 08:39:30.509029 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.509008 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7fxz"] Apr 16 08:39:30.599488 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599441 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.599680 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599504 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fe087373-6d9b-4de6-8388-7fca0f0df84b-klusterlet-config\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.599680 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599541 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.599680 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599597 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.599680 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599633 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.599680 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599665 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48959348-ef25-4f02-8521-38b7e30ce7a1-tmp-dir\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599697 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe087373-6d9b-4de6-8388-7fca0f0df84b-tmp\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599732 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599767 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599795 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599827 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/7b622291-ca24-445f-8aca-ac06f0bef275-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599856 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjq5h\" (UniqueName: \"kubernetes.io/projected/48959348-ef25-4f02-8521-38b7e30ce7a1-kube-api-access-hjq5h\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599889 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599896 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ps6df"] Apr 16 08:39:30.599922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599919 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599945 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.599978 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jz2f\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600008 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvd8b\" (UniqueName: \"kubernetes.io/projected/370f20de-694e-4186-8f44-08ff3e69991c-kube-api-access-gvd8b\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600045 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbrx8\" (UniqueName: \"kubernetes.io/projected/fe087373-6d9b-4de6-8388-7fca0f0df84b-kube-api-access-zbrx8\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600099 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48959348-ef25-4f02-8521-38b7e30ce7a1-config-volume\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600135 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600159 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600189 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370f20de-694e-4186-8f44-08ff3e69991c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.600341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.600240 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcmgw\" (UniqueName: \"kubernetes.io/projected/7b622291-ca24-445f-8aca-ac06f0bef275-kube-api-access-hcmgw\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.603890 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.603869 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.605739 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.605717 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Apr 16 08:39:30.605858 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.605842 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Apr 16 08:39:30.605916 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.605873 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q9sj7\"" Apr 16 08:39:30.605968 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.605948 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Apr 16 08:39:30.607948 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.607926 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ps6df"] Apr 16 08:39:30.701146 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701114 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.701146 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701152 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fe087373-6d9b-4de6-8388-7fca0f0df84b-klusterlet-config\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.701393 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701178 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.701393 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701210 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.701393 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701235 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.701393 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701266 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.701947 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701913 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48959348-ef25-4f02-8521-38b7e30ce7a1-tmp-dir\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.702087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701967 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe087373-6d9b-4de6-8388-7fca0f0df84b-tmp\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.702087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.701998 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.702087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702026 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.702087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702073 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702097 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/7b622291-ca24-445f-8aca-ac06f0bef275-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702122 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hjq5h\" (UniqueName: \"kubernetes.io/projected/48959348-ef25-4f02-8521-38b7e30ce7a1-kube-api-access-hjq5h\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702153 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702180 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702206 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702236 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5jz2f\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702237 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48959348-ef25-4f02-8521-38b7e30ce7a1-tmp-dir\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702262 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gvd8b\" (UniqueName: \"kubernetes.io/projected/370f20de-694e-4186-8f44-08ff3e69991c-kube-api-access-gvd8b\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.702309 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702299 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbrx8\" (UniqueName: \"kubernetes.io/projected/fe087373-6d9b-4de6-8388-7fca0f0df84b-kube-api-access-zbrx8\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.702706 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702314 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe087373-6d9b-4de6-8388-7fca0f0df84b-tmp\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.702706 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702327 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44hmg\" (UniqueName: \"kubernetes.io/projected/c63ff564-0d80-4740-8ca8-39f1950fe05a-kube-api-access-44hmg\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.703236 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.702986 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/7b622291-ca24-445f-8aca-ac06f0bef275-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.703236 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.703217 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:30.703463 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.703304 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:30.703463 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.703322 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:30.703463 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703337 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.703463 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703347 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48959348-ef25-4f02-8521-38b7e30ce7a1-config-volume\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.703463 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.703383 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:31.203363261 +0000 UTC m=+34.565375183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:30.703710 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.703557 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:39:31.203413893 +0000 UTC m=+34.565425802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:30.703710 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703644 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.703710 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703678 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.703710 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703706 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370f20de-694e-4186-8f44-08ff3e69991c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.703979 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703751 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hcmgw\" (UniqueName: \"kubernetes.io/projected/7b622291-ca24-445f-8aca-ac06f0bef275-kube-api-access-hcmgw\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.703979 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.703829 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.704348 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.704320 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48959348-ef25-4f02-8521-38b7e30ce7a1-config-volume\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.704651 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.704537 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.706848 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.707423 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fe087373-6d9b-4de6-8388-7fca0f0df84b-klusterlet-config\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.707443 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.707568 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-hub\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.707838 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-ca\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.708167 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.708756 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7b622291-ca24-445f-8aca-ac06f0bef275-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.708921 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/370f20de-694e-4186-8f44-08ff3e69991c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.710181 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjq5h\" (UniqueName: \"kubernetes.io/projected/48959348-ef25-4f02-8521-38b7e30ce7a1-kube-api-access-hjq5h\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.711688 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jz2f\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.711951 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvd8b\" (UniqueName: \"kubernetes.io/projected/370f20de-694e-4186-8f44-08ff3e69991c-kube-api-access-gvd8b\") pod \"managed-serviceaccount-addon-agent-69d57c7545-gldxx\" (UID: \"370f20de-694e-4186-8f44-08ff3e69991c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.713077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.712761 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:30.713733 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.713187 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcmgw\" (UniqueName: \"kubernetes.io/projected/7b622291-ca24-445f-8aca-ac06f0bef275-kube-api-access-hcmgw\") pod \"cluster-proxy-proxy-agent-654b77b445-d5n5h\" (UID: \"7b622291-ca24-445f-8aca-ac06f0bef275\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.713733 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.713212 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbrx8\" (UniqueName: \"kubernetes.io/projected/fe087373-6d9b-4de6-8388-7fca0f0df84b-kube-api-access-zbrx8\") pod \"klusterlet-addon-workmgr-65675cb4bd-v7drg\" (UID: \"fe087373-6d9b-4de6-8388-7fca0f0df84b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.804588 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.804504 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-44hmg\" (UniqueName: \"kubernetes.io/projected/c63ff564-0d80-4740-8ca8-39f1950fe05a-kube-api-access-44hmg\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.804751 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.804606 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.804751 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.804727 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:30.804913 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.804788 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:39:31.30477394 +0000 UTC m=+34.666785865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:30.812719 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.812695 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44hmg\" (UniqueName: \"kubernetes.io/projected/c63ff564-0d80-4740-8ca8-39f1950fe05a-kube-api-access-44hmg\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:30.814369 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.814346 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:30.826225 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.826202 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:39:30.834834 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.834818 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" Apr 16 08:39:30.905812 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:30.905770 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:30.905925 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.905884 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:30.905971 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:30.905943 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:02.905928959 +0000 UTC m=+66.267940871 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : object "openshift-multus"/"metrics-daemon-secret" not registered Apr 16 08:39:31.108706 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.108667 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:31.108886 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.108864 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Apr 16 08:39:31.108966 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.108888 2569 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Apr 16 08:39:31.108966 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.108902 2569 projected.go:194] Error preparing data for projected volume kube-api-access-88s45 for pod openshift-network-diagnostics/network-check-target-jqclg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:31.109106 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.108966 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45 podName:7315ebc3-7fa9-49f2-be69-7b9d6f0e5934 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:03.108945591 +0000 UTC m=+66.470957521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-88s45" (UniqueName: "kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45") pod "network-check-target-jqclg" (UID: "7315ebc3-7fa9-49f2-be69-7b9d6f0e5934") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Apr 16 08:39:31.209489 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.209439 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:31.209702 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.209508 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:31.209702 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.209654 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:31.209820 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.209712 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:32.209698957 +0000 UTC m=+35.571710865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:31.209820 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.209656 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:31.209820 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.209745 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:31.209820 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.209794 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:39:32.209780402 +0000 UTC m=+35.571792310 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:31.271957 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.271891 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx"] Apr 16 08:39:31.272794 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.272768 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h"] Apr 16 08:39:31.275363 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.275117 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg"] Apr 16 08:39:31.310354 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.310326 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:31.310485 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.310469 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:31.310538 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:31.310529 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:39:32.310514679 +0000 UTC m=+35.672526587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:31.320566 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:39:31.320535 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod370f20de_694e_4186_8f44_08ff3e69991c.slice/crio-7e16b2e6f51ed583c51159ccddc79395d835be3dc7e84463e47cdf9920ca1c02 WatchSource:0}: Error finding container 7e16b2e6f51ed583c51159ccddc79395d835be3dc7e84463e47cdf9920ca1c02: Status 404 returned error can't find the container with id 7e16b2e6f51ed583c51159ccddc79395d835be3dc7e84463e47cdf9920ca1c02 Apr 16 08:39:31.321295 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:39:31.321225 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b622291_ca24_445f_8aca_ac06f0bef275.slice/crio-8e885cf656cccb72f8b161971c2e5783ade4bc4afcde462be6e5c8f5d1e1c207 WatchSource:0}: Error finding container 8e885cf656cccb72f8b161971c2e5783ade4bc4afcde462be6e5c8f5d1e1c207: Status 404 returned error can't find the container with id 8e885cf656cccb72f8b161971c2e5783ade4bc4afcde462be6e5c8f5d1e1c207 Apr 16 08:39:31.322042 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:39:31.322020 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe087373_6d9b_4de6_8388_7fca0f0df84b.slice/crio-86ccf4658929baf4c3958e92087233a4b450eecea9240149dda7680674997567 WatchSource:0}: Error finding container 86ccf4658929baf4c3958e92087233a4b450eecea9240149dda7680674997567: Status 404 returned error can't find the container with id 86ccf4658929baf4c3958e92087233a4b450eecea9240149dda7680674997567 Apr 16 08:39:31.417985 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.417952 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" event={"ID":"370f20de-694e-4186-8f44-08ff3e69991c","Type":"ContainerStarted","Data":"7e16b2e6f51ed583c51159ccddc79395d835be3dc7e84463e47cdf9920ca1c02"} Apr 16 08:39:31.419564 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.419536 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" event={"ID":"fe087373-6d9b-4de6-8388-7fca0f0df84b","Type":"ContainerStarted","Data":"86ccf4658929baf4c3958e92087233a4b450eecea9240149dda7680674997567"} Apr 16 08:39:31.420587 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:31.420567 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerStarted","Data":"8e885cf656cccb72f8b161971c2e5783ade4bc4afcde462be6e5c8f5d1e1c207"} Apr 16 08:39:32.218146 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.218106 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:32.218331 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.218267 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:32.218331 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.218283 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:32.218331 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.218307 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:32.218484 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.218372 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:39:34.218351444 +0000 UTC m=+37.580363354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:32.218484 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.218380 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:32.218484 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.218430 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:34.218412594 +0000 UTC m=+37.580424509 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:32.252562 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.252532 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:39:32.252562 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.252554 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:32.252882 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.252861 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.256334 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.256588 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdwck\"" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.256777 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.256989 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9gvlt\"" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.257194 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 08:39:32.257649 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.257413 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Apr 16 08:39:32.318918 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.318859 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:32.319404 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.319018 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:32.319404 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:32.319106 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:39:34.31908796 +0000 UTC m=+37.681099873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:32.433599 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.432430 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="7967b9f3cd30098e275992ed0c7dc5a6ad22f61676cbec98dabac299b13883bc" exitCode=0 Apr 16 08:39:32.433599 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:32.432498 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"7967b9f3cd30098e275992ed0c7dc5a6ad22f61676cbec98dabac299b13883bc"} Apr 16 08:39:33.439373 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:33.439336 2569 generic.go:358] "Generic (PLEG): container finished" podID="3759b22a-834b-4461-859a-8702206c5593" containerID="8be9abe6b50d8ecfbedd5f81feedc0de8ab7da9247b0f9fd56ae538b9016fb6b" exitCode=0 Apr 16 08:39:33.439796 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:33.439399 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerDied","Data":"8be9abe6b50d8ecfbedd5f81feedc0de8ab7da9247b0f9fd56ae538b9016fb6b"} Apr 16 08:39:34.238927 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:34.238847 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:34.239087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:34.238992 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:34.239087 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.239004 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:34.239087 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.239025 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:34.239203 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.239095 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:39:38.239079408 +0000 UTC m=+41.601091317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:34.239203 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.239137 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:34.239308 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.239203 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:38.23918654 +0000 UTC m=+41.601198450 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:34.340457 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:34.340357 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:34.340603 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.340533 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:34.340670 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:34.340614 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:39:38.340590768 +0000 UTC m=+41.702602688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:37.450635 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.450544 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" event={"ID":"3759b22a-834b-4461-859a-8702206c5593","Type":"ContainerStarted","Data":"bd1ebd2a73452794d8f83973f98f3ba8946c536290f064944d570b509c565e74"} Apr 16 08:39:37.452090 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.452040 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerStarted","Data":"6d6dc37b992b7f2527b6ddaf9ee0ca5ca0e51ad60977ad2ccd9793a38b8bb3a1"} Apr 16 08:39:37.453357 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.453331 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" event={"ID":"370f20de-694e-4186-8f44-08ff3e69991c","Type":"ContainerStarted","Data":"9c7f355b846aeed241674f868f69259770fb579c0cc3a370e0ff280be3689526"} Apr 16 08:39:37.454591 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.454572 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" event={"ID":"fe087373-6d9b-4de6-8388-7fca0f0df84b","Type":"ContainerStarted","Data":"e84bce361fcbcea24057876843abe5dd6084b7193db49c92f7dacc8d9a92731f"} Apr 16 08:39:37.454799 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.454781 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:37.456341 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.456325 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:39:37.469648 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.469613 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rzw5p" podStartSLOduration=9.050539065 podStartE2EDuration="40.469589854s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:38:59.934586154 +0000 UTC m=+3.296598075" lastFinishedPulling="2026-04-16 08:39:31.353636956 +0000 UTC m=+34.715648864" observedRunningTime="2026-04-16 08:39:37.469351335 +0000 UTC m=+40.831363265" watchObservedRunningTime="2026-04-16 08:39:37.469589854 +0000 UTC m=+40.831601783" Apr 16 08:39:37.483230 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.483193 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" podStartSLOduration=10.643023697 podStartE2EDuration="16.483182385s" podCreationTimestamp="2026-04-16 08:39:21 +0000 UTC" firstStartedPulling="2026-04-16 08:39:31.32921467 +0000 UTC m=+34.691226581" lastFinishedPulling="2026-04-16 08:39:37.16937336 +0000 UTC m=+40.531385269" observedRunningTime="2026-04-16 08:39:37.48208999 +0000 UTC m=+40.844101919" watchObservedRunningTime="2026-04-16 08:39:37.483182385 +0000 UTC m=+40.845194315" Apr 16 08:39:37.496588 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:37.496546 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" podStartSLOduration=10.656680955 podStartE2EDuration="16.496535732s" podCreationTimestamp="2026-04-16 08:39:21 +0000 UTC" firstStartedPulling="2026-04-16 08:39:31.329500621 +0000 UTC m=+34.691512530" lastFinishedPulling="2026-04-16 08:39:37.169355393 +0000 UTC m=+40.531367307" observedRunningTime="2026-04-16 08:39:37.495883149 +0000 UTC m=+40.857895080" watchObservedRunningTime="2026-04-16 08:39:37.496535732 +0000 UTC m=+40.858547656" Apr 16 08:39:38.273173 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:38.273134 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:38.273321 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:38.273252 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:38.273321 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.273309 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:38.273411 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.273331 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:38.273411 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.273355 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:38.273477 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.273413 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:39:46.273393608 +0000 UTC m=+49.635405535 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:38.273477 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.273434 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:39:46.273424409 +0000 UTC m=+49.635436322 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:38.374382 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:38.374316 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:38.374543 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.374457 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:38.374543 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:38.374515 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:39:46.374499679 +0000 UTC m=+49.736511603 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:40.462319 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.462283 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerStarted","Data":"8a2e7287f8c3c15b1d50497c811808b67e2a971560a6da455c0235e77682a88e"} Apr 16 08:39:40.462668 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.462326 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerStarted","Data":"56cb5b8edb21cd72f0fa41f8b340a727b842fbfb60825c181a3dcae4d4dc516b"} Apr 16 08:39:40.480495 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.480454 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" podStartSLOduration=10.837538651 podStartE2EDuration="19.480440019s" podCreationTimestamp="2026-04-16 08:39:21 +0000 UTC" firstStartedPulling="2026-04-16 08:39:31.329273963 +0000 UTC m=+34.691285874" lastFinishedPulling="2026-04-16 08:39:39.972175316 +0000 UTC m=+43.334187242" observedRunningTime="2026-04-16 08:39:40.479742733 +0000 UTC m=+43.841754662" watchObservedRunningTime="2026-04-16 08:39:40.480440019 +0000 UTC m=+43.842452148" Apr 16 08:39:40.690382 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.690345 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:40.693814 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.693786 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1597cea1-f09d-46f2-9513-72dde7873326-original-pull-secret\") pod \"global-pull-secret-syncer-4vp8m\" (UID: \"1597cea1-f09d-46f2-9513-72dde7873326\") " pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:40.993615 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:40.993576 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4vp8m" Apr 16 08:39:41.105391 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:41.105352 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4vp8m"] Apr 16 08:39:41.107838 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:39:41.107803 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1597cea1_f09d_46f2_9513_72dde7873326.slice/crio-ae6dbc479580e5f0cd060497cfc1ea1c0a83abee0f82e0b58bfdc46a3e78010a WatchSource:0}: Error finding container ae6dbc479580e5f0cd060497cfc1ea1c0a83abee0f82e0b58bfdc46a3e78010a: Status 404 returned error can't find the container with id ae6dbc479580e5f0cd060497cfc1ea1c0a83abee0f82e0b58bfdc46a3e78010a Apr 16 08:39:41.465396 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:41.465356 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vp8m" event={"ID":"1597cea1-f09d-46f2-9513-72dde7873326","Type":"ContainerStarted","Data":"ae6dbc479580e5f0cd060497cfc1ea1c0a83abee0f82e0b58bfdc46a3e78010a"} Apr 16 08:39:45.476424 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:45.476383 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4vp8m" event={"ID":"1597cea1-f09d-46f2-9513-72dde7873326","Type":"ContainerStarted","Data":"fd5151bb091d9858d9e14cd574bd309c287cfe722a54cbdcd24c30961029dcb3"} Apr 16 08:39:45.489273 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:45.489227 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4vp8m" podStartSLOduration=33.590851577 podStartE2EDuration="37.489213002s" podCreationTimestamp="2026-04-16 08:39:08 +0000 UTC" firstStartedPulling="2026-04-16 08:39:41.109655574 +0000 UTC m=+44.471667481" lastFinishedPulling="2026-04-16 08:39:45.008016994 +0000 UTC m=+48.370028906" observedRunningTime="2026-04-16 08:39:45.488791174 +0000 UTC m=+48.850803104" watchObservedRunningTime="2026-04-16 08:39:45.489213002 +0000 UTC m=+48.851224932" Apr 16 08:39:46.325809 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:46.325769 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:39:46.325976 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:46.325862 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:39:46.325976 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.325902 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:39:46.325976 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.325921 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:39:46.325976 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.325942 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:39:46.325976 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.325973 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:40:02.325958406 +0000 UTC m=+65.687970315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:39:46.326156 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.325995 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:02.32598447 +0000 UTC m=+65.687996378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:39:46.426789 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:46.426755 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:39:46.426895 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.426865 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:39:46.426930 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:39:46.426920 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:40:02.426906872 +0000 UTC m=+65.788918780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:39:56.417022 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:39:56.416997 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xkwdc" Apr 16 08:40:02.347567 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:02.347513 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:40:02.347567 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:02.347573 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:40:02.347996 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.347657 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:40:02.347996 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.347656 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:40:02.347996 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.347667 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:40:02.347996 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.347719 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:40:34.347705827 +0000 UTC m=+97.709717735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:40:02.347996 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.347731 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:40:34.34772601 +0000 UTC m=+97.709737918 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:40:02.448070 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:02.448017 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:40:02.448222 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.448155 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:40:02.448222 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.448215 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:40:34.448200047 +0000 UTC m=+97.810211954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:40:02.951914 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:02.951871 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:40:02.953797 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:02.953778 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Apr 16 08:40:02.962309 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.962281 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 08:40:02.962373 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:02.962354 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:06.962332057 +0000 UTC m=+130.324343969 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : secret "metrics-daemon-secret" not found Apr 16 08:40:03.153699 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.153666 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:40:03.155702 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.155679 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Apr 16 08:40:03.165614 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.165587 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Apr 16 08:40:03.177577 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.177556 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88s45\" (UniqueName: \"kubernetes.io/projected/7315ebc3-7fa9-49f2-be69-7b9d6f0e5934-kube-api-access-88s45\") pod \"network-check-target-jqclg\" (UID: \"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934\") " pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:40:03.203519 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.203462 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zdwck\"" Apr 16 08:40:03.212838 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.212818 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:40:03.324407 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.324376 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jqclg"] Apr 16 08:40:03.327480 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:40:03.327454 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7315ebc3_7fa9_49f2_be69_7b9d6f0e5934.slice/crio-5122ae68182fa314e8c68230e27db1105996fc126df9ac9bca25795df6fd3555 WatchSource:0}: Error finding container 5122ae68182fa314e8c68230e27db1105996fc126df9ac9bca25795df6fd3555: Status 404 returned error can't find the container with id 5122ae68182fa314e8c68230e27db1105996fc126df9ac9bca25795df6fd3555 Apr 16 08:40:03.519887 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:03.519807 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqclg" event={"ID":"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934","Type":"ContainerStarted","Data":"5122ae68182fa314e8c68230e27db1105996fc126df9ac9bca25795df6fd3555"} Apr 16 08:40:06.528731 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:06.528650 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jqclg" event={"ID":"7315ebc3-7fa9-49f2-be69-7b9d6f0e5934","Type":"ContainerStarted","Data":"76ce4c56ca96029506bb6cf302ece404ff8a549f1590a8c7b124e1765e8df3c4"} Apr 16 08:40:06.529107 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:06.528775 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:40:06.542595 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:06.542552 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jqclg" podStartSLOduration=66.618023676 podStartE2EDuration="1m9.542539901s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:40:03.329383818 +0000 UTC m=+66.691395726" lastFinishedPulling="2026-04-16 08:40:06.253900043 +0000 UTC m=+69.615911951" observedRunningTime="2026-04-16 08:40:06.541966205 +0000 UTC m=+69.903978139" watchObservedRunningTime="2026-04-16 08:40:06.542539901 +0000 UTC m=+69.904551844" Apr 16 08:40:34.384398 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:34.384276 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:40:34.384398 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:34.384340 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:40:34.384884 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.384449 2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Apr 16 08:40:34.384884 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.384475 2569 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Apr 16 08:40:34.384884 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.384487 2569 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-566d6b79cc-dklnq: secret "image-registry-tls" not found Apr 16 08:40:34.384884 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.384549 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls podName:403f70d7-0e93-4536-9f55-4009c733748b nodeName:}" failed. No retries permitted until 2026-04-16 08:41:38.384532709 +0000 UTC m=+161.746544635 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls") pod "image-registry-566d6b79cc-dklnq" (UID: "403f70d7-0e93-4536-9f55-4009c733748b") : secret "image-registry-tls" not found Apr 16 08:40:34.384884 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.384563 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls podName:48959348-ef25-4f02-8521-38b7e30ce7a1 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:38.384556493 +0000 UTC m=+161.746568401 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls") pod "dns-default-h7fxz" (UID: "48959348-ef25-4f02-8521-38b7e30ce7a1") : secret "dns-default-metrics-tls" not found Apr 16 08:40:34.485494 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:34.485432 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:40:34.485661 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.485570 2569 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Apr 16 08:40:34.485661 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:40:34.485640 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert podName:c63ff564-0d80-4740-8ca8-39f1950fe05a nodeName:}" failed. No retries permitted until 2026-04-16 08:41:38.485615359 +0000 UTC m=+161.847627267 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert") pod "ingress-canary-ps6df" (UID: "c63ff564-0d80-4740-8ca8-39f1950fe05a") : secret "canary-serving-cert" not found Apr 16 08:40:37.533775 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:40:37.533735 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jqclg" Apr 16 08:41:06.685884 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:06.685856 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k2pss_0ee13c74-b0d0-42a3-959b-963a8f04c0b0/dns-node-resolver/0.log" Apr 16 08:41:07.018366 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:07.018265 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:41:07.018537 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:07.018400 2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Apr 16 08:41:07.018537 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:07.018477 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs podName:1b29b84e-b6c8-46e1-891b-df2585f55350 nodeName:}" failed. No retries permitted until 2026-04-16 08:43:09.018455422 +0000 UTC m=+252.380467347 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs") pod "network-metrics-daemon-mcq96" (UID: "1b29b84e-b6c8-46e1-891b-df2585f55350") : secret "metrics-daemon-secret" not found Apr 16 08:41:07.286359 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:07.286287 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7gcs5_5a824f8c-b47c-4092-8fa7-c2564991fd54/node-ca/0.log" Apr 16 08:41:25.950864 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.950831 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-22kr6"] Apr 16 08:41:25.953877 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.953854 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.955603 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.955576 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Apr 16 08:41:25.956162 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.956146 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Apr 16 08:41:25.956272 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.956157 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Apr 16 08:41:25.956272 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.956178 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Apr 16 08:41:25.956272 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.956214 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jjxz9\"" Apr 16 08:41:25.962115 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.962098 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/516340c2-5ce8-4e39-8636-3b98afc987b1-data-volume\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.962200 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.962143 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/516340c2-5ce8-4e39-8636-3b98afc987b1-crio-socket\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.962200 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.962191 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/516340c2-5ce8-4e39-8636-3b98afc987b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.962320 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.962305 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.962357 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.962330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77qn5\" (UniqueName: \"kubernetes.io/projected/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-api-access-77qn5\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:25.966217 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:25.966197 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-22kr6"] Apr 16 08:41:26.062735 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.062699 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/516340c2-5ce8-4e39-8636-3b98afc987b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.062922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.062776 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.062922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.062805 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77qn5\" (UniqueName: \"kubernetes.io/projected/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-api-access-77qn5\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.062922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.062849 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/516340c2-5ce8-4e39-8636-3b98afc987b1-data-volume\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.063131 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.062958 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/516340c2-5ce8-4e39-8636-3b98afc987b1-crio-socket\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.063131 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.063033 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/516340c2-5ce8-4e39-8636-3b98afc987b1-crio-socket\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.063239 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.063218 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/516340c2-5ce8-4e39-8636-3b98afc987b1-data-volume\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.063386 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.063369 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.065166 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.065146 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/516340c2-5ce8-4e39-8636-3b98afc987b1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.086201 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.086174 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77qn5\" (UniqueName: \"kubernetes.io/projected/516340c2-5ce8-4e39-8636-3b98afc987b1-kube-api-access-77qn5\") pod \"insights-runtime-extractor-22kr6\" (UID: \"516340c2-5ce8-4e39-8636-3b98afc987b1\") " pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.262229 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.262142 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-22kr6" Apr 16 08:41:26.380677 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.380647 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-22kr6"] Apr 16 08:41:26.383722 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:41:26.383693 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516340c2_5ce8_4e39_8636_3b98afc987b1.slice/crio-11eab047fb453661c3441220dc544037e1bb61eded22fb92a1c5574459f9f4f0 WatchSource:0}: Error finding container 11eab047fb453661c3441220dc544037e1bb61eded22fb92a1c5574459f9f4f0: Status 404 returned error can't find the container with id 11eab047fb453661c3441220dc544037e1bb61eded22fb92a1c5574459f9f4f0 Apr 16 08:41:26.721770 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.721731 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22kr6" event={"ID":"516340c2-5ce8-4e39-8636-3b98afc987b1","Type":"ContainerStarted","Data":"246d26d4627cdeb4821181dd8f0d7bd852c5765fc980a062a91f5aaf2cdb6ff4"} Apr 16 08:41:26.721770 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:26.721768 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22kr6" event={"ID":"516340c2-5ce8-4e39-8636-3b98afc987b1","Type":"ContainerStarted","Data":"11eab047fb453661c3441220dc544037e1bb61eded22fb92a1c5574459f9f4f0"} Apr 16 08:41:27.726529 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:27.726484 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22kr6" event={"ID":"516340c2-5ce8-4e39-8636-3b98afc987b1","Type":"ContainerStarted","Data":"7bef58675f7657d079652554a821ea946468b975774aa02654e1f5891cd46769"} Apr 16 08:41:28.730790 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:28.730750 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-22kr6" event={"ID":"516340c2-5ce8-4e39-8636-3b98afc987b1","Type":"ContainerStarted","Data":"5e918c96868147a8ecaf20773e1bf58fa995e89be73a4348f4e418924de7374f"} Apr 16 08:41:33.503558 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:33.503516 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" podUID="403f70d7-0e93-4536-9f55-4009c733748b" Apr 16 08:41:33.539762 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:33.539726 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-h7fxz" podUID="48959348-ef25-4f02-8521-38b7e30ce7a1" Apr 16 08:41:33.624334 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:33.624296 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-ps6df" podUID="c63ff564-0d80-4740-8ca8-39f1950fe05a" Apr 16 08:41:33.744354 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:33.744323 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:41:33.744503 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:33.744328 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:33.744503 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:33.744328 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:41:35.269360 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:35.269323 2569 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mcq96" podUID="1b29b84e-b6c8-46e1-891b-df2585f55350" Apr 16 08:41:37.455650 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.455596 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" podUID="fe087373-6d9b-4de6-8388-7fca0f0df84b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.9:8000/readyz\": dial tcp 10.132.0.9:8000: connect: connection refused" Apr 16 08:41:37.756016 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.755986 2569 generic.go:358] "Generic (PLEG): container finished" podID="370f20de-694e-4186-8f44-08ff3e69991c" containerID="9c7f355b846aeed241674f868f69259770fb579c0cc3a370e0ff280be3689526" exitCode=255 Apr 16 08:41:37.756202 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.756078 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" event={"ID":"370f20de-694e-4186-8f44-08ff3e69991c","Type":"ContainerDied","Data":"9c7f355b846aeed241674f868f69259770fb579c0cc3a370e0ff280be3689526"} Apr 16 08:41:37.756449 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.756430 2569 scope.go:117] "RemoveContainer" containerID="9c7f355b846aeed241674f868f69259770fb579c0cc3a370e0ff280be3689526" Apr 16 08:41:37.757365 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.757345 2569 generic.go:358] "Generic (PLEG): container finished" podID="fe087373-6d9b-4de6-8388-7fca0f0df84b" containerID="e84bce361fcbcea24057876843abe5dd6084b7193db49c92f7dacc8d9a92731f" exitCode=1 Apr 16 08:41:37.757439 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.757380 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" event={"ID":"fe087373-6d9b-4de6-8388-7fca0f0df84b","Type":"ContainerDied","Data":"e84bce361fcbcea24057876843abe5dd6084b7193db49c92f7dacc8d9a92731f"} Apr 16 08:41:37.757714 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.757698 2569 scope.go:117] "RemoveContainer" containerID="e84bce361fcbcea24057876843abe5dd6084b7193db49c92f7dacc8d9a92731f" Apr 16 08:41:37.771625 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:37.771575 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-22kr6" podStartSLOduration=10.720628143999999 podStartE2EDuration="12.771561599s" podCreationTimestamp="2026-04-16 08:41:25 +0000 UTC" firstStartedPulling="2026-04-16 08:41:26.438110443 +0000 UTC m=+149.800122351" lastFinishedPulling="2026-04-16 08:41:28.489043898 +0000 UTC m=+151.851055806" observedRunningTime="2026-04-16 08:41:28.746746399 +0000 UTC m=+152.108758328" watchObservedRunningTime="2026-04-16 08:41:37.771561599 +0000 UTC m=+161.133573529" Apr 16 08:41:38.455471 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.455416 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:41:38.455657 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.455521 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:38.457937 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.457906 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48959348-ef25-4f02-8521-38b7e30ce7a1-metrics-tls\") pod \"dns-default-h7fxz\" (UID: \"48959348-ef25-4f02-8521-38b7e30ce7a1\") " pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:38.458251 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.457986 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"image-registry-566d6b79cc-dklnq\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:41:38.548835 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.548802 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9d2hv\"" Apr 16 08:41:38.548978 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.548802 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-v9nxr\"" Apr 16 08:41:38.556332 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.556313 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:38.556422 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.556403 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:41:38.556604 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.556583 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:41:38.559092 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.559073 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c63ff564-0d80-4740-8ca8-39f1950fe05a-cert\") pod \"ingress-canary-ps6df\" (UID: \"c63ff564-0d80-4740-8ca8-39f1950fe05a\") " pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:41:38.682734 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.682704 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:41:38.685722 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:41:38.685692 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403f70d7_0e93_4536_9f55_4009c733748b.slice/crio-1e8facc2c16f52b1a026c52130debfa52219997be01428da172a4ec9d558d7f9 WatchSource:0}: Error finding container 1e8facc2c16f52b1a026c52130debfa52219997be01428da172a4ec9d558d7f9: Status 404 returned error can't find the container with id 1e8facc2c16f52b1a026c52130debfa52219997be01428da172a4ec9d558d7f9 Apr 16 08:41:38.696005 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.695974 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-h7fxz"] Apr 16 08:41:38.701333 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:41:38.701308 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48959348_ef25_4f02_8521_38b7e30ce7a1.slice/crio-e26ab8e46981b1aab58fcea9f27430436f1e4d4d5b88313293d60ac065969dae WatchSource:0}: Error finding container e26ab8e46981b1aab58fcea9f27430436f1e4d4d5b88313293d60ac065969dae: Status 404 returned error can't find the container with id e26ab8e46981b1aab58fcea9f27430436f1e4d4d5b88313293d60ac065969dae Apr 16 08:41:38.760900 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.760870 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" event={"ID":"fe087373-6d9b-4de6-8388-7fca0f0df84b","Type":"ContainerStarted","Data":"4a8698e2cb0bc26c8de824bf933aba4e2b57fed64cebbc20bea0fc85131b245a"} Apr 16 08:41:38.761337 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.761171 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:41:38.762660 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.762639 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-65675cb4bd-v7drg" Apr 16 08:41:38.762758 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.762670 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7fxz" event={"ID":"48959348-ef25-4f02-8521-38b7e30ce7a1","Type":"ContainerStarted","Data":"e26ab8e46981b1aab58fcea9f27430436f1e4d4d5b88313293d60ac065969dae"} Apr 16 08:41:38.764288 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.764267 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-69d57c7545-gldxx" event={"ID":"370f20de-694e-4186-8f44-08ff3e69991c","Type":"ContainerStarted","Data":"ba333fa63b2369e6357758f4cef5f381fd28abee35b2e7b73662517567ee785c"} Apr 16 08:41:38.765495 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.765475 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" event={"ID":"403f70d7-0e93-4536-9f55-4009c733748b","Type":"ContainerStarted","Data":"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7"} Apr 16 08:41:38.765594 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.765501 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" event={"ID":"403f70d7-0e93-4536-9f55-4009c733748b","Type":"ContainerStarted","Data":"1e8facc2c16f52b1a026c52130debfa52219997be01428da172a4ec9d558d7f9"} Apr 16 08:41:38.765645 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.765609 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:41:38.791920 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.791863 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" podStartSLOduration=161.791845444 podStartE2EDuration="2m41.791845444s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:41:38.791038598 +0000 UTC m=+162.153050529" watchObservedRunningTime="2026-04-16 08:41:38.791845444 +0000 UTC m=+162.153857375" Apr 16 08:41:38.847200 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.847156 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-q9sj7\"" Apr 16 08:41:38.856524 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.856503 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ps6df" Apr 16 08:41:38.974590 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:38.974501 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ps6df"] Apr 16 08:41:38.977567 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:41:38.977533 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc63ff564_0d80_4740_8ca8_39f1950fe05a.slice/crio-a11c4d2426e305d7d7a452fa3a57ba7dd14e4337a34c68e645ec2a762c90c085 WatchSource:0}: Error finding container a11c4d2426e305d7d7a452fa3a57ba7dd14e4337a34c68e645ec2a762c90c085: Status 404 returned error can't find the container with id a11c4d2426e305d7d7a452fa3a57ba7dd14e4337a34c68e645ec2a762c90c085 Apr 16 08:41:39.769700 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:39.769628 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ps6df" event={"ID":"c63ff564-0d80-4740-8ca8-39f1950fe05a","Type":"ContainerStarted","Data":"a11c4d2426e305d7d7a452fa3a57ba7dd14e4337a34c68e645ec2a762c90c085"} Apr 16 08:41:40.774046 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:40.774012 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ps6df" event={"ID":"c63ff564-0d80-4740-8ca8-39f1950fe05a","Type":"ContainerStarted","Data":"7e145799564ba75af7e3b21b0d8676b0294fe767512a401d5264f3538071f069"} Apr 16 08:41:40.775843 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:40.775818 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7fxz" event={"ID":"48959348-ef25-4f02-8521-38b7e30ce7a1","Type":"ContainerStarted","Data":"3577bfde803c7bcbf5be1ec7d96f1803b5b2e191c7bf5dbf12e0bcb6ba4dde2b"} Apr 16 08:41:40.788922 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:40.788844 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ps6df" podStartSLOduration=129.186413263 podStartE2EDuration="2m10.788829327s" podCreationTimestamp="2026-04-16 08:39:30 +0000 UTC" firstStartedPulling="2026-04-16 08:41:38.979278022 +0000 UTC m=+162.341289930" lastFinishedPulling="2026-04-16 08:41:40.58169408 +0000 UTC m=+163.943705994" observedRunningTime="2026-04-16 08:41:40.788222574 +0000 UTC m=+164.150234505" watchObservedRunningTime="2026-04-16 08:41:40.788829327 +0000 UTC m=+164.150841275" Apr 16 08:41:41.779725 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:41.779686 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-h7fxz" event={"ID":"48959348-ef25-4f02-8521-38b7e30ce7a1","Type":"ContainerStarted","Data":"150da79479a5d54f6de2ae37d38d326071772569b9041dd5687db9b506c0246d"} Apr 16 08:41:41.794183 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:41.794130 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-h7fxz" podStartSLOduration=129.958661335 podStartE2EDuration="2m11.794113392s" podCreationTimestamp="2026-04-16 08:39:30 +0000 UTC" firstStartedPulling="2026-04-16 08:41:38.703205895 +0000 UTC m=+162.065217804" lastFinishedPulling="2026-04-16 08:41:40.538657953 +0000 UTC m=+163.900669861" observedRunningTime="2026-04-16 08:41:41.793672636 +0000 UTC m=+165.155684578" watchObservedRunningTime="2026-04-16 08:41:41.794113392 +0000 UTC m=+165.156125325" Apr 16 08:41:42.178515 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.178480 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-srh26"] Apr 16 08:41:42.181563 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.181546 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.183249 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183224 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Apr 16 08:41:42.183512 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183496 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Apr 16 08:41:42.183620 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183586 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Apr 16 08:41:42.183832 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183819 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lc2hb\"" Apr 16 08:41:42.183889 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183861 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Apr 16 08:41:42.183939 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183912 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Apr 16 08:41:42.183991 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.183912 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Apr 16 08:41:42.287556 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287525 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287556 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287559 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm4qc\" (UniqueName: \"kubernetes.io/projected/723d8931-2012-4f4b-91ed-79a2994a8f92-kube-api-access-sm4qc\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287580 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-textfile\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287609 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-wtmp\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287642 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-sys\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287658 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-metrics-client-ca\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287686 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-root\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287767 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287755 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.287947 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.287792 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-accelerators-collector-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388661 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388617 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-accelerators-collector-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388661 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388667 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388688 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sm4qc\" (UniqueName: \"kubernetes.io/projected/723d8931-2012-4f4b-91ed-79a2994a8f92-kube-api-access-sm4qc\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388707 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-textfile\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388729 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-wtmp\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388759 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-sys\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388778 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-metrics-client-ca\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:42.388778 2569 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388800 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-root\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388841 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.388876 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:41:42.388856 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls podName:723d8931-2012-4f4b-91ed-79a2994a8f92 nodeName:}" failed. No retries permitted until 2026-04-16 08:41:42.888835727 +0000 UTC m=+166.250847644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls") pod "node-exporter-srh26" (UID: "723d8931-2012-4f4b-91ed-79a2994a8f92") : secret "node-exporter-tls" not found Apr 16 08:41:42.389232 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388918 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-wtmp\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.389232 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388854 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-sys\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.389232 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.388984 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/723d8931-2012-4f4b-91ed-79a2994a8f92-root\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.389232 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.389213 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-textfile\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.389360 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.389301 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-accelerators-collector-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.389403 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.389364 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/723d8931-2012-4f4b-91ed-79a2994a8f92-metrics-client-ca\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.391283 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.391266 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.396190 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.396158 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm4qc\" (UniqueName: \"kubernetes.io/projected/723d8931-2012-4f4b-91ed-79a2994a8f92-kube-api-access-sm4qc\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.782556 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.782523 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:42.892973 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.892931 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:42.895394 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:42.895376 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/723d8931-2012-4f4b-91ed-79a2994a8f92-node-exporter-tls\") pod \"node-exporter-srh26\" (UID: \"723d8931-2012-4f4b-91ed-79a2994a8f92\") " pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:43.091318 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:43.091239 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srh26" Apr 16 08:41:43.101657 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:41:43.101632 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723d8931_2012_4f4b_91ed_79a2994a8f92.slice/crio-8320785284a2f7ab8214024a1ede65486c88f0338a9d6d7d2b20cdda9c923769 WatchSource:0}: Error finding container 8320785284a2f7ab8214024a1ede65486c88f0338a9d6d7d2b20cdda9c923769: Status 404 returned error can't find the container with id 8320785284a2f7ab8214024a1ede65486c88f0338a9d6d7d2b20cdda9c923769 Apr 16 08:41:43.786647 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:43.786609 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srh26" event={"ID":"723d8931-2012-4f4b-91ed-79a2994a8f92","Type":"ContainerStarted","Data":"8320785284a2f7ab8214024a1ede65486c88f0338a9d6d7d2b20cdda9c923769"} Apr 16 08:41:44.790789 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:44.790759 2569 generic.go:358] "Generic (PLEG): container finished" podID="723d8931-2012-4f4b-91ed-79a2994a8f92" containerID="db402fe5d7c28086f2ab87b61fac3e9a0b25867d7160fee17d40d73f38470f7c" exitCode=0 Apr 16 08:41:44.791171 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:44.790831 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srh26" event={"ID":"723d8931-2012-4f4b-91ed-79a2994a8f92","Type":"ContainerDied","Data":"db402fe5d7c28086f2ab87b61fac3e9a0b25867d7160fee17d40d73f38470f7c"} Apr 16 08:41:45.798458 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:45.798422 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srh26" event={"ID":"723d8931-2012-4f4b-91ed-79a2994a8f92","Type":"ContainerStarted","Data":"47b310d07b2fc9201815aac11c0f872685334d648b9e72a013d7c6856952664b"} Apr 16 08:41:45.798458 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:45.798459 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srh26" event={"ID":"723d8931-2012-4f4b-91ed-79a2994a8f92","Type":"ContainerStarted","Data":"acf5994d6db83b5460802ae6f64f9d81e4a6f75f0d356264d34beead458360b4"} Apr 16 08:41:45.815025 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:45.814980 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-srh26" podStartSLOduration=3.033833844 podStartE2EDuration="3.814966825s" podCreationTimestamp="2026-04-16 08:41:42 +0000 UTC" firstStartedPulling="2026-04-16 08:41:43.103869971 +0000 UTC m=+166.465881890" lastFinishedPulling="2026-04-16 08:41:43.885002963 +0000 UTC m=+167.247014871" observedRunningTime="2026-04-16 08:41:45.813898544 +0000 UTC m=+169.175910475" watchObservedRunningTime="2026-04-16 08:41:45.814966825 +0000 UTC m=+169.176978754" Apr 16 08:41:48.252354 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:48.252319 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:41:52.789563 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:52.789527 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-h7fxz" Apr 16 08:41:58.230847 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:58.230811 2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:41:58.235179 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:58.235148 2569 patch_prober.go:28] interesting pod/image-registry-566d6b79cc-dklnq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Apr 16 08:41:58.235305 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:41:58.235201 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" podUID="403f70d7-0e93-4536-9f55-4009c733748b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 16 08:42:08.235822 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:08.235793 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:42:10.827290 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:10.827253 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" podUID="7b622291-ca24-445f-8aca-ac06f0bef275" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 08:42:15.073126 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:15.073093 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ps6df_c63ff564-0d80-4740-8ca8-39f1950fe05a/serve-healthcheck-canary/0.log" Apr 16 08:42:20.828034 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:20.827995 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" podUID="7b622291-ca24-445f-8aca-ac06f0bef275" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 08:42:23.249999 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.249931 2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" podUID="403f70d7-0e93-4536-9f55-4009c733748b" containerName="registry" containerID="cri-o://38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7" gracePeriod=30 Apr 16 08:42:23.484859 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.484837 2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:42:23.608657 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608572 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608657 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608620 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608657 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608640 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608887 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608674 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jz2f\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608887 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608793 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608887 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608837 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.608887 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608868 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.609118 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.608904 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets\") pod \"403f70d7-0e93-4536-9f55-4009c733748b\" (UID: \"403f70d7-0e93-4536-9f55-4009c733748b\") " Apr 16 08:42:23.609292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.609234 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:23.609895 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.609863 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Apr 16 08:42:23.611388 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.611357 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:42:23.611523 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.611486 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:42:23.611523 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.611496 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f" (OuterVolumeSpecName: "kube-api-access-5jz2f") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "kube-api-access-5jz2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:42:23.611723 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.611706 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:23.611790 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.611767 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Apr 16 08:42:23.619566 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.619543 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "403f70d7-0e93-4536-9f55-4009c733748b" (UID: "403f70d7-0e93-4536-9f55-4009c733748b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Apr 16 08:42:23.710153 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710129 2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-installation-pull-secrets\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710153 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710149 2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/403f70d7-0e93-4536-9f55-4009c733748b-ca-trust-extracted\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710159 2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-registry-tls\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710170 2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/403f70d7-0e93-4536-9f55-4009c733748b-image-registry-private-configuration\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710180 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jz2f\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-kube-api-access-5jz2f\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710190 2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-registry-certificates\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710198 2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/403f70d7-0e93-4536-9f55-4009c733748b-bound-sa-token\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.710292 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.710206 2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/403f70d7-0e93-4536-9f55-4009c733748b-trusted-ca\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:42:23.903111 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.903072 2569 generic.go:358] "Generic (PLEG): container finished" podID="403f70d7-0e93-4536-9f55-4009c733748b" containerID="38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7" exitCode=0 Apr 16 08:42:23.903277 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.903154 2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" Apr 16 08:42:23.903277 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.903149 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" event={"ID":"403f70d7-0e93-4536-9f55-4009c733748b","Type":"ContainerDied","Data":"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7"} Apr 16 08:42:23.903277 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.903255 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-566d6b79cc-dklnq" event={"ID":"403f70d7-0e93-4536-9f55-4009c733748b","Type":"ContainerDied","Data":"1e8facc2c16f52b1a026c52130debfa52219997be01428da172a4ec9d558d7f9"} Apr 16 08:42:23.903277 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.903274 2569 scope.go:117] "RemoveContainer" containerID="38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7" Apr 16 08:42:23.912701 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.912680 2569 scope.go:117] "RemoveContainer" containerID="38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7" Apr 16 08:42:23.912969 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:42:23.912947 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7\": container with ID starting with 38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7 not found: ID does not exist" containerID="38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7" Apr 16 08:42:23.913078 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.912973 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7"} err="failed to get container status \"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7\": rpc error: code = NotFound desc = could not find container \"38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7\": container with ID starting with 38103705f48de610f7d238f2fc2228c6b414b5b5cb7b5ae7931988d35e8a70b7 not found: ID does not exist" Apr 16 08:42:23.922450 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.922429 2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:42:23.926192 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:23.926171 2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-566d6b79cc-dklnq"] Apr 16 08:42:25.257949 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:25.257915 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="403f70d7-0e93-4536-9f55-4009c733748b" path="/var/lib/kubelet/pods/403f70d7-0e93-4536-9f55-4009c733748b/volumes" Apr 16 08:42:30.827132 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:30.827090 2569 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" podUID="7b622291-ca24-445f-8aca-ac06f0bef275" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Apr 16 08:42:30.827662 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:30.827170 2569 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" Apr 16 08:42:30.827662 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:30.827593 2569 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8a2e7287f8c3c15b1d50497c811808b67e2a971560a6da455c0235e77682a88e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" containerMessage="Container service-proxy failed liveness probe, will be restarted" Apr 16 08:42:30.827662 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:30.827626 2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" podUID="7b622291-ca24-445f-8aca-ac06f0bef275" containerName="service-proxy" containerID="cri-o://8a2e7287f8c3c15b1d50497c811808b67e2a971560a6da455c0235e77682a88e" gracePeriod=30 Apr 16 08:42:31.926690 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:31.926654 2569 generic.go:358] "Generic (PLEG): container finished" podID="7b622291-ca24-445f-8aca-ac06f0bef275" containerID="8a2e7287f8c3c15b1d50497c811808b67e2a971560a6da455c0235e77682a88e" exitCode=2 Apr 16 08:42:31.927078 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:31.926723 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerDied","Data":"8a2e7287f8c3c15b1d50497c811808b67e2a971560a6da455c0235e77682a88e"} Apr 16 08:42:31.927078 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:42:31.926758 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-654b77b445-d5n5h" event={"ID":"7b622291-ca24-445f-8aca-ac06f0bef275","Type":"ContainerStarted","Data":"a1c6b166176208be79e95d2b8bdd38c87a090d6994edb957b391d40018d3f5cd"} Apr 16 08:43:09.022739 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:09.022664 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:43:09.025004 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:09.024985 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b29b84e-b6c8-46e1-891b-df2585f55350-metrics-certs\") pod \"network-metrics-daemon-mcq96\" (UID: \"1b29b84e-b6c8-46e1-891b-df2585f55350\") " pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:43:09.255261 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:09.255232 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9gvlt\"" Apr 16 08:43:09.263730 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:09.263709 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mcq96" Apr 16 08:43:09.396903 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:09.396876 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mcq96"] Apr 16 08:43:09.400272 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:43:09.400245 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b29b84e_b6c8_46e1_891b_df2585f55350.slice/crio-f0fa4d4a8db6fc8efd0991de4131ae31379a6b8453a2646aa7909c7edb08bc04 WatchSource:0}: Error finding container f0fa4d4a8db6fc8efd0991de4131ae31379a6b8453a2646aa7909c7edb08bc04: Status 404 returned error can't find the container with id f0fa4d4a8db6fc8efd0991de4131ae31379a6b8453a2646aa7909c7edb08bc04 Apr 16 08:43:10.021995 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:10.021961 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcq96" event={"ID":"1b29b84e-b6c8-46e1-891b-df2585f55350","Type":"ContainerStarted","Data":"f0fa4d4a8db6fc8efd0991de4131ae31379a6b8453a2646aa7909c7edb08bc04"} Apr 16 08:43:11.029195 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:11.029152 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcq96" event={"ID":"1b29b84e-b6c8-46e1-891b-df2585f55350","Type":"ContainerStarted","Data":"8589048ff39fca1214ba15fd56b51b587224c3c755b14e647eb3e61669d6d59e"} Apr 16 08:43:11.029195 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:11.029196 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mcq96" event={"ID":"1b29b84e-b6c8-46e1-891b-df2585f55350","Type":"ContainerStarted","Data":"3e559b7f64839d1fa2fda7114030cced5dbfcf36bb7c2cf31c2d005b35ae66de"} Apr 16 08:43:11.043258 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:11.043203 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mcq96" podStartSLOduration=253.062135657 podStartE2EDuration="4m14.043183655s" podCreationTimestamp="2026-04-16 08:38:57 +0000 UTC" firstStartedPulling="2026-04-16 08:43:09.40218228 +0000 UTC m=+252.764194203" lastFinishedPulling="2026-04-16 08:43:10.383230284 +0000 UTC m=+253.745242201" observedRunningTime="2026-04-16 08:43:11.041956446 +0000 UTC m=+254.403968375" watchObservedRunningTime="2026-04-16 08:43:11.043183655 +0000 UTC m=+254.405195586" Apr 16 08:43:57.152859 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:57.152834 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:43:57.154100 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:57.154075 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:43:57.165263 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:43:57.165244 2569 kubelet.go:1628] "Image garbage collection succeeded" Apr 16 08:44:54.349178 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.349147 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-qshzg"] Apr 16 08:44:54.349582 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.349359 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403f70d7-0e93-4536-9f55-4009c733748b" containerName="registry" Apr 16 08:44:54.349582 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.349369 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="403f70d7-0e93-4536-9f55-4009c733748b" containerName="registry" Apr 16 08:44:54.349582 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.349411 2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="403f70d7-0e93-4536-9f55-4009c733748b" containerName="registry" Apr 16 08:44:54.352000 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.351981 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.353958 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.353937 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-qjgtg\"" Apr 16 08:44:54.354076 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.353966 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Apr 16 08:44:54.354076 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.354015 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Apr 16 08:44:54.358597 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.358569 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-qshzg"] Apr 16 08:44:54.417358 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.417329 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kp95\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-kube-api-access-2kp95\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.417358 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.417370 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.518654 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.518617 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2kp95\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-kube-api-access-2kp95\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.518820 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.518663 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.526061 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.526025 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-bound-sa-token\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.526215 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.526195 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kp95\" (UniqueName: \"kubernetes.io/projected/a923c20d-20c0-470b-bff8-14a63350ff6b-kube-api-access-2kp95\") pod \"cert-manager-cainjector-8966b78d4-qshzg\" (UID: \"a923c20d-20c0-470b-bff8-14a63350ff6b\") " pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.661397 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.661360 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" Apr 16 08:44:54.776644 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.776623 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-8966b78d4-qshzg"] Apr 16 08:44:54.778979 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:44:54.778950 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda923c20d_20c0_470b_bff8_14a63350ff6b.slice/crio-cd455ea1de91fa7b431cb78a1fd5e36638a11982c255aab4e0061ab46bfb67e0 WatchSource:0}: Error finding container cd455ea1de91fa7b431cb78a1fd5e36638a11982c255aab4e0061ab46bfb67e0: Status 404 returned error can't find the container with id cd455ea1de91fa7b431cb78a1fd5e36638a11982c255aab4e0061ab46bfb67e0 Apr 16 08:44:54.785338 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:54.781093 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 08:44:55.298025 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:55.297986 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" event={"ID":"a923c20d-20c0-470b-bff8-14a63350ff6b","Type":"ContainerStarted","Data":"cd455ea1de91fa7b431cb78a1fd5e36638a11982c255aab4e0061ab46bfb67e0"} Apr 16 08:44:58.310509 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:58.310417 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" event={"ID":"a923c20d-20c0-470b-bff8-14a63350ff6b","Type":"ContainerStarted","Data":"091a2c5bd6b972c9b1ff7f5e18fcd4dfb0d4f68d55d81e26bf3bad7fe395c2d6"} Apr 16 08:44:58.324213 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:44:58.324165 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-8966b78d4-qshzg" podStartSLOduration=1.089973621 podStartE2EDuration="4.324151077s" podCreationTimestamp="2026-04-16 08:44:54 +0000 UTC" firstStartedPulling="2026-04-16 08:44:54.781301609 +0000 UTC m=+358.143313524" lastFinishedPulling="2026-04-16 08:44:58.015479072 +0000 UTC m=+361.377490980" observedRunningTime="2026-04-16 08:44:58.323850322 +0000 UTC m=+361.685862256" watchObservedRunningTime="2026-04-16 08:44:58.324151077 +0000 UTC m=+361.686163007" Apr 16 08:45:01.480627 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.480591 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-759f64656b-4w6rb"] Apr 16 08:45:01.483851 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.483828 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.485708 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.485691 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-srwv7\"" Apr 16 08:45:01.490526 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.490506 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-759f64656b-4w6rb"] Apr 16 08:45:01.572325 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.572284 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-bound-sa-token\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.572508 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.572354 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phpc6\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-kube-api-access-phpc6\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.672793 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.672760 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-phpc6\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-kube-api-access-phpc6\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.672793 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.672800 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-bound-sa-token\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.680189 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.680159 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phpc6\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-kube-api-access-phpc6\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.680189 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.680163 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aad07e1e-6268-4d9e-8f1e-048d1a78dde8-bound-sa-token\") pod \"cert-manager-759f64656b-4w6rb\" (UID: \"aad07e1e-6268-4d9e-8f1e-048d1a78dde8\") " pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.793087 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.792991 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-759f64656b-4w6rb" Apr 16 08:45:01.908808 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:01.908779 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-759f64656b-4w6rb"] Apr 16 08:45:01.911630 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:45:01.911603 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaad07e1e_6268_4d9e_8f1e_048d1a78dde8.slice/crio-218a3355df9e1eebe23cce51f9c298b109739f8702e36592ebd3ea66bb2c61f0 WatchSource:0}: Error finding container 218a3355df9e1eebe23cce51f9c298b109739f8702e36592ebd3ea66bb2c61f0: Status 404 returned error can't find the container with id 218a3355df9e1eebe23cce51f9c298b109739f8702e36592ebd3ea66bb2c61f0 Apr 16 08:45:02.322350 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:02.322317 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-759f64656b-4w6rb" event={"ID":"aad07e1e-6268-4d9e-8f1e-048d1a78dde8","Type":"ContainerStarted","Data":"876742d2272b581a921fa594a66e2057f9b18a7027865f48d0ae5578e30d96f9"} Apr 16 08:45:02.322350 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:02.322350 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-759f64656b-4w6rb" event={"ID":"aad07e1e-6268-4d9e-8f1e-048d1a78dde8","Type":"ContainerStarted","Data":"218a3355df9e1eebe23cce51f9c298b109739f8702e36592ebd3ea66bb2c61f0"} Apr 16 08:45:02.336012 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:02.335939 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-759f64656b-4w6rb" podStartSLOduration=1.335923452 podStartE2EDuration="1.335923452s" podCreationTimestamp="2026-04-16 08:45:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:45:02.335653586 +0000 UTC m=+365.697665527" watchObservedRunningTime="2026-04-16 08:45:02.335923452 +0000 UTC m=+365.697935389" Apr 16 08:45:19.129435 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.129394 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8"] Apr 16 08:45:19.132582 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.132551 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.134743 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.134720 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Apr 16 08:45:19.134743 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.134733 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k8z2v\"" Apr 16 08:45:19.134910 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.134724 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Apr 16 08:45:19.135044 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.135017 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Apr 16 08:45:19.135155 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.135022 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Apr 16 08:45:19.142357 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.142337 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8"] Apr 16 08:45:19.196361 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.196330 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wjpj\" (UniqueName: \"kubernetes.io/projected/f3286354-03df-4614-990c-249fbe6bd7f3-kube-api-access-9wjpj\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.196555 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.196382 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.196555 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.196449 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.297731 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.297696 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.297881 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.297748 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9wjpj\" (UniqueName: \"kubernetes.io/projected/f3286354-03df-4614-990c-249fbe6bd7f3-kube-api-access-9wjpj\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.297881 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.297783 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.300317 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.300286 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-webhook-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.300317 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.300311 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f3286354-03df-4614-990c-249fbe6bd7f3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.304909 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.304888 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wjpj\" (UniqueName: \"kubernetes.io/projected/f3286354-03df-4614-990c-249fbe6bd7f3-kube-api-access-9wjpj\") pod \"opendatahub-operator-controller-manager-569944d57d-wlnj8\" (UID: \"f3286354-03df-4614-990c-249fbe6bd7f3\") " pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.443493 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.443457 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:19.564871 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:19.564848 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8"] Apr 16 08:45:19.567501 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:45:19.567472 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3286354_03df_4614_990c_249fbe6bd7f3.slice/crio-9b6fa480da335e8dde6b8cfcf57a6f580c9c91f0c6708c56e92b2cf7c84ceed4 WatchSource:0}: Error finding container 9b6fa480da335e8dde6b8cfcf57a6f580c9c91f0c6708c56e92b2cf7c84ceed4: Status 404 returned error can't find the container with id 9b6fa480da335e8dde6b8cfcf57a6f580c9c91f0c6708c56e92b2cf7c84ceed4 Apr 16 08:45:20.370080 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:20.370007 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" event={"ID":"f3286354-03df-4614-990c-249fbe6bd7f3","Type":"ContainerStarted","Data":"9b6fa480da335e8dde6b8cfcf57a6f580c9c91f0c6708c56e92b2cf7c84ceed4"} Apr 16 08:45:22.376312 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:22.376215 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" event={"ID":"f3286354-03df-4614-990c-249fbe6bd7f3","Type":"ContainerStarted","Data":"40508a8dc334acae4ec72edbe3285c753c8f8e235a676c8e1792f96af26f70f5"} Apr 16 08:45:22.376645 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:22.376322 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:22.393827 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:22.393784 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" podStartSLOduration=0.965380705 podStartE2EDuration="3.393771487s" podCreationTimestamp="2026-04-16 08:45:19 +0000 UTC" firstStartedPulling="2026-04-16 08:45:19.569176729 +0000 UTC m=+382.931188641" lastFinishedPulling="2026-04-16 08:45:21.997567502 +0000 UTC m=+385.359579423" observedRunningTime="2026-04-16 08:45:22.392318529 +0000 UTC m=+385.754330458" watchObservedRunningTime="2026-04-16 08:45:22.393771487 +0000 UTC m=+385.755783418" Apr 16 08:45:33.380933 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:33.380907 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-569944d57d-wlnj8" Apr 16 08:45:40.857243 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.857207 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7ffb6"] Apr 16 08:45:40.860120 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.860104 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:40.861884 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.861866 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Apr 16 08:45:40.861978 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.861881 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-lcmvl\"" Apr 16 08:45:40.869843 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.869824 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7ffb6"] Apr 16 08:45:40.944393 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.944367 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:40.944522 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:40.944402 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxh48\" (UniqueName: \"kubernetes.io/projected/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-kube-api-access-nxh48\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:41.045666 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:41.045635 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:41.045666 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:41.045668 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nxh48\" (UniqueName: \"kubernetes.io/projected/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-kube-api-access-nxh48\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:41.045842 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:41.045785 2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Apr 16 08:45:41.045878 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:41.045845 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert podName:fc3c99a7-e248-47f6-9ef2-b554a48c4a8d nodeName:}" failed. No retries permitted until 2026-04-16 08:45:41.545829999 +0000 UTC m=+404.907841908 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert") pod "odh-model-controller-858dbf95b8-7ffb6" (UID: "fc3c99a7-e248-47f6-9ef2-b554a48c4a8d") : secret "odh-model-controller-webhook-cert" not found Apr 16 08:45:41.053706 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:41.053688 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxh48\" (UniqueName: \"kubernetes.io/projected/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-kube-api-access-nxh48\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:41.551105 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:41.551073 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:41.551284 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:41.551146 2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Apr 16 08:45:41.551284 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:41.551209 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert podName:fc3c99a7-e248-47f6-9ef2-b554a48c4a8d nodeName:}" failed. No retries permitted until 2026-04-16 08:45:42.551194393 +0000 UTC m=+405.913206301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert") pod "odh-model-controller-858dbf95b8-7ffb6" (UID: "fc3c99a7-e248-47f6-9ef2-b554a48c4a8d") : secret "odh-model-controller-webhook-cert" not found Apr 16 08:45:42.559947 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:42.559910 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:42.562552 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:42.562528 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc3c99a7-e248-47f6-9ef2-b554a48c4a8d-cert\") pod \"odh-model-controller-858dbf95b8-7ffb6\" (UID: \"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d\") " pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:42.669670 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:42.669630 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:42.787476 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:42.787444 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7ffb6"] Apr 16 08:45:42.790683 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:45:42.790657 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc3c99a7_e248_47f6_9ef2_b554a48c4a8d.slice/crio-1473608a625c5ad5084ad0b32e523c0f3318b346ece3a2d369169a4e005f9fce WatchSource:0}: Error finding container 1473608a625c5ad5084ad0b32e523c0f3318b346ece3a2d369169a4e005f9fce: Status 404 returned error can't find the container with id 1473608a625c5ad5084ad0b32e523c0f3318b346ece3a2d369169a4e005f9fce Apr 16 08:45:43.436236 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:43.436196 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" event={"ID":"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d","Type":"ContainerStarted","Data":"1473608a625c5ad5084ad0b32e523c0f3318b346ece3a2d369169a4e005f9fce"} Apr 16 08:45:45.443001 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:45.442955 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" event={"ID":"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d","Type":"ContainerStarted","Data":"fe8eeffa1639d74e2595d282ad3795a62fbbe99ab2d4fa3df7329b98f5ab3fb4"} Apr 16 08:45:45.443420 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:45.443120 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:45.460301 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:45.460251 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" podStartSLOduration=2.971888948 podStartE2EDuration="5.460231736s" podCreationTimestamp="2026-04-16 08:45:40 +0000 UTC" firstStartedPulling="2026-04-16 08:45:42.792299165 +0000 UTC m=+406.154311073" lastFinishedPulling="2026-04-16 08:45:45.280641943 +0000 UTC m=+408.642653861" observedRunningTime="2026-04-16 08:45:45.458256121 +0000 UTC m=+408.820268050" watchObservedRunningTime="2026-04-16 08:45:45.460231736 +0000 UTC m=+408.822243667" Apr 16 08:45:46.154648 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.154621 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-qhwqx"] Apr 16 08:45:46.157858 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.157841 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.159515 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.159486 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Apr 16 08:45:46.159629 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.159493 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kjxnh\"" Apr 16 08:45:46.165878 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.165837 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-qhwqx"] Apr 16 08:45:46.294415 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.294384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frrb8\" (UniqueName: \"kubernetes.io/projected/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-kube-api-access-frrb8\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.294579 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.294429 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-cert\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.395219 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.395192 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frrb8\" (UniqueName: \"kubernetes.io/projected/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-kube-api-access-frrb8\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.395369 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.395232 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-cert\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.397724 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.397694 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-cert\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.405320 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.405261 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frrb8\" (UniqueName: \"kubernetes.io/projected/deee6c74-ed99-49c0-86e0-ad7cd7fe230f-kube-api-access-frrb8\") pod \"kserve-controller-manager-856948b99f-qhwqx\" (UID: \"deee6c74-ed99-49c0-86e0-ad7cd7fe230f\") " pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.446799 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.446770 2569 generic.go:358] "Generic (PLEG): container finished" podID="fc3c99a7-e248-47f6-9ef2-b554a48c4a8d" containerID="fe8eeffa1639d74e2595d282ad3795a62fbbe99ab2d4fa3df7329b98f5ab3fb4" exitCode=1 Apr 16 08:45:46.447162 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.446812 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" event={"ID":"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d","Type":"ContainerDied","Data":"fe8eeffa1639d74e2595d282ad3795a62fbbe99ab2d4fa3df7329b98f5ab3fb4"} Apr 16 08:45:46.447162 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.447069 2569 scope.go:117] "RemoveContainer" containerID="fe8eeffa1639d74e2595d282ad3795a62fbbe99ab2d4fa3df7329b98f5ab3fb4" Apr 16 08:45:46.470460 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.470435 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:46.587128 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:46.587096 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-qhwqx"] Apr 16 08:45:46.590629 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:45:46.590596 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddeee6c74_ed99_49c0_86e0_ad7cd7fe230f.slice/crio-649d76a1ebc261d447efb958763a082bbc62fd5571e601064d48ebfd003ce472 WatchSource:0}: Error finding container 649d76a1ebc261d447efb958763a082bbc62fd5571e601064d48ebfd003ce472: Status 404 returned error can't find the container with id 649d76a1ebc261d447efb958763a082bbc62fd5571e601064d48ebfd003ce472 Apr 16 08:45:47.205104 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.205064 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7"] Apr 16 08:45:47.208187 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.208163 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.210480 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.210450 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Apr 16 08:45:47.210614 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.210485 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Apr 16 08:45:47.210614 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.210502 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Apr 16 08:45:47.210614 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.210526 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Apr 16 08:45:47.220161 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.220139 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7"] Apr 16 08:45:47.300975 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.300943 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.301145 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.300979 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzs7q\" (UniqueName: \"kubernetes.io/projected/e0a114ba-6f14-49f3-b21e-394243bb6f59-kube-api-access-vzs7q\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.301145 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.301099 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0a114ba-6f14-49f3-b21e-394243bb6f59-tmp\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.402421 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.402374 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.402421 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.402423 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vzs7q\" (UniqueName: \"kubernetes.io/projected/e0a114ba-6f14-49f3-b21e-394243bb6f59-kube-api-access-vzs7q\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.402716 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.402484 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0a114ba-6f14-49f3-b21e-394243bb6f59-tmp\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.402716 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:47.402512 2569 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Apr 16 08:45:47.402716 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:45:47.402593 2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs podName:e0a114ba-6f14-49f3-b21e-394243bb6f59 nodeName:}" failed. No retries permitted until 2026-04-16 08:45:47.902570788 +0000 UTC m=+411.264582716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs") pod "kube-auth-proxy-5d5746998b-cxkd7" (UID: "e0a114ba-6f14-49f3-b21e-394243bb6f59") : secret "kube-auth-proxy-tls" not found Apr 16 08:45:47.404816 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.404794 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0a114ba-6f14-49f3-b21e-394243bb6f59-tmp\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.411896 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.411877 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzs7q\" (UniqueName: \"kubernetes.io/projected/e0a114ba-6f14-49f3-b21e-394243bb6f59-kube-api-access-vzs7q\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.451621 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.451571 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" event={"ID":"fc3c99a7-e248-47f6-9ef2-b554a48c4a8d","Type":"ContainerStarted","Data":"322f20249f884e8a0d1f10254e78f868cdb3bebbc31cca767ff34dc3732df436"} Apr 16 08:45:47.452102 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.451768 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:45:47.453068 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.453025 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" event={"ID":"deee6c74-ed99-49c0-86e0-ad7cd7fe230f","Type":"ContainerStarted","Data":"649d76a1ebc261d447efb958763a082bbc62fd5571e601064d48ebfd003ce472"} Apr 16 08:45:47.907101 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.907039 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:47.909664 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:47.909641 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0a114ba-6f14-49f3-b21e-394243bb6f59-tls-certs\") pod \"kube-auth-proxy-5d5746998b-cxkd7\" (UID: \"e0a114ba-6f14-49f3-b21e-394243bb6f59\") " pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:48.117224 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:48.117189 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" Apr 16 08:45:48.239913 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:48.239877 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7"] Apr 16 08:45:48.242558 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:45:48.242525 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a114ba_6f14_49f3_b21e_394243bb6f59.slice/crio-11a6b55158ae262314768605d56d98f9f836cec2c4ca0976372f1ac85401a98b WatchSource:0}: Error finding container 11a6b55158ae262314768605d56d98f9f836cec2c4ca0976372f1ac85401a98b: Status 404 returned error can't find the container with id 11a6b55158ae262314768605d56d98f9f836cec2c4ca0976372f1ac85401a98b Apr 16 08:45:48.457304 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:48.457215 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" event={"ID":"e0a114ba-6f14-49f3-b21e-394243bb6f59","Type":"ContainerStarted","Data":"11a6b55158ae262314768605d56d98f9f836cec2c4ca0976372f1ac85401a98b"} Apr 16 08:45:49.462786 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:49.462733 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" event={"ID":"deee6c74-ed99-49c0-86e0-ad7cd7fe230f","Type":"ContainerStarted","Data":"ba8d6a1fe712f4e83ea9933f22b0f3c43b2414f152a22fb9276092467a30d911"} Apr 16 08:45:49.463296 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:49.462962 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:45:49.478312 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:49.477101 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" podStartSLOduration=1.006510589 podStartE2EDuration="3.477084227s" podCreationTimestamp="2026-04-16 08:45:46 +0000 UTC" firstStartedPulling="2026-04-16 08:45:46.591962718 +0000 UTC m=+409.953974626" lastFinishedPulling="2026-04-16 08:45:49.062536353 +0000 UTC m=+412.424548264" observedRunningTime="2026-04-16 08:45:49.476632972 +0000 UTC m=+412.838644915" watchObservedRunningTime="2026-04-16 08:45:49.477084227 +0000 UTC m=+412.839096156" Apr 16 08:45:52.476668 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:52.476633 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" event={"ID":"e0a114ba-6f14-49f3-b21e-394243bb6f59","Type":"ContainerStarted","Data":"b77fe6ab4c39a87d0b8ff290e63c5047eac7714eb14cfc401a65c4bc85924196"} Apr 16 08:45:52.490135 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:52.490079 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d5746998b-cxkd7" podStartSLOduration=1.9086854450000001 podStartE2EDuration="5.490041082s" podCreationTimestamp="2026-04-16 08:45:47 +0000 UTC" firstStartedPulling="2026-04-16 08:45:48.244143605 +0000 UTC m=+411.606155516" lastFinishedPulling="2026-04-16 08:45:51.82549924 +0000 UTC m=+415.187511153" observedRunningTime="2026-04-16 08:45:52.489420879 +0000 UTC m=+415.851432809" watchObservedRunningTime="2026-04-16 08:45:52.490041082 +0000 UTC m=+415.852053013" Apr 16 08:45:58.460015 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:45:58.459986 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-7ffb6" Apr 16 08:46:02.175871 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.175822 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-n9js6"] Apr 16 08:46:02.184327 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.184306 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.186300 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.186276 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Apr 16 08:46:02.186406 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.186316 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-txfm7\"" Apr 16 08:46:02.186406 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.186288 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Apr 16 08:46:02.189944 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.189918 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-n9js6"] Apr 16 08:46:02.318956 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.318928 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p66c\" (UniqueName: \"kubernetes.io/projected/884324ce-110b-4544-b0aa-ff1fe4a3c21c-kube-api-access-2p66c\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.319153 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.318973 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/884324ce-110b-4544-b0aa-ff1fe4a3c21c-operator-config\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.420319 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.420270 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2p66c\" (UniqueName: \"kubernetes.io/projected/884324ce-110b-4544-b0aa-ff1fe4a3c21c-kube-api-access-2p66c\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.420519 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.420344 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/884324ce-110b-4544-b0aa-ff1fe4a3c21c-operator-config\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.423023 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.422997 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/884324ce-110b-4544-b0aa-ff1fe4a3c21c-operator-config\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.435412 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.435347 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p66c\" (UniqueName: \"kubernetes.io/projected/884324ce-110b-4544-b0aa-ff1fe4a3c21c-kube-api-access-2p66c\") pod \"servicemesh-operator3-55f49c5f94-n9js6\" (UID: \"884324ce-110b-4544-b0aa-ff1fe4a3c21c\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.493957 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.493927 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:02.613460 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:02.613402 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-n9js6"] Apr 16 08:46:02.616606 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:46:02.616581 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884324ce_110b_4544_b0aa_ff1fe4a3c21c.slice/crio-6352f4e61159a14fd564ef128ae7d5f5f429a73ad7ba05c58fffc377337b0353 WatchSource:0}: Error finding container 6352f4e61159a14fd564ef128ae7d5f5f429a73ad7ba05c58fffc377337b0353: Status 404 returned error can't find the container with id 6352f4e61159a14fd564ef128ae7d5f5f429a73ad7ba05c58fffc377337b0353 Apr 16 08:46:03.507337 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:03.507299 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" event={"ID":"884324ce-110b-4544-b0aa-ff1fe4a3c21c","Type":"ContainerStarted","Data":"6352f4e61159a14fd564ef128ae7d5f5f429a73ad7ba05c58fffc377337b0353"} Apr 16 08:46:05.516463 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:05.516425 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" event={"ID":"884324ce-110b-4544-b0aa-ff1fe4a3c21c","Type":"ContainerStarted","Data":"fb90a0482151373f72564e19d312216277fa90927dbd681294775fdcc8664396"} Apr 16 08:46:05.516871 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:05.516551 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:05.536710 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:05.536665 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" podStartSLOduration=1.389327818 podStartE2EDuration="3.536652273s" podCreationTimestamp="2026-04-16 08:46:02 +0000 UTC" firstStartedPulling="2026-04-16 08:46:02.619132276 +0000 UTC m=+425.981144184" lastFinishedPulling="2026-04-16 08:46:04.766456727 +0000 UTC m=+428.128468639" observedRunningTime="2026-04-16 08:46:05.53626178 +0000 UTC m=+428.898273709" watchObservedRunningTime="2026-04-16 08:46:05.536652273 +0000 UTC m=+428.898664203" Apr 16 08:46:16.526103 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:16.526019 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-n9js6" Apr 16 08:46:17.433718 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.433677 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf"] Apr 16 08:46:17.439662 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.439638 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440221 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440196 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440337 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440232 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d200eb10-0d27-49e0-807e-37a41ee4a1cb-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440337 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440275 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440337 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440328 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440437 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440352 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440437 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440390 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhn97\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-kube-api-access-qhn97\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.440437 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.440408 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.442523 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.442504 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"istio-ca-root-cert\"" Apr 16 08:46:17.442523 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.442511 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wdxdp\"" Apr 16 08:46:17.442523 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.442509 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Apr 16 08:46:17.442768 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.442638 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Apr 16 08:46:17.442840 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.442826 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Apr 16 08:46:17.456785 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.456762 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf"] Apr 16 08:46:17.541000 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.540972 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541000 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541009 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541481 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541031 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541481 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541075 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qhn97\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-kube-api-access-qhn97\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541481 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541096 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541481 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541122 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541481 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541143 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d200eb10-0d27-49e0-807e-37a41ee4a1cb-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.541695 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.541646 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.543712 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.543677 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-cacerts\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.543712 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.543711 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-kubeconfig\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.543985 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.543969 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d200eb10-0d27-49e0-807e-37a41ee4a1cb-local-certs\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.544144 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.544125 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.553301 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.553280 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-istio-token\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.553463 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.553437 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhn97\" (UniqueName: \"kubernetes.io/projected/d200eb10-0d27-49e0-807e-37a41ee4a1cb-kube-api-access-qhn97\") pod \"istiod-openshift-gateway-55ff986f96-bbtrf\" (UID: \"d200eb10-0d27-49e0-807e-37a41ee4a1cb\") " pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.749422 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.749335 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:17.889910 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:17.889880 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf"] Apr 16 08:46:17.891682 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:46:17.891642 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd200eb10_0d27_49e0_807e_37a41ee4a1cb.slice/crio-d819e9f067929a7d6b48f55999c4803352361662c13a00729e247e38bd7c7a13 WatchSource:0}: Error finding container d819e9f067929a7d6b48f55999c4803352361662c13a00729e247e38bd7c7a13: Status 404 returned error can't find the container with id d819e9f067929a7d6b48f55999c4803352361662c13a00729e247e38bd7c7a13 Apr 16 08:46:18.558221 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:18.558192 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" event={"ID":"d200eb10-0d27-49e0-807e-37a41ee4a1cb","Type":"ContainerStarted","Data":"d819e9f067929a7d6b48f55999c4803352361662c13a00729e247e38bd7c7a13"} Apr 16 08:46:20.474559 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:20.474529 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-qhwqx" Apr 16 08:46:21.664459 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:21.664397 2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Apr 16 08:46:21.664744 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:21.664490 2569 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Apr 16 08:46:22.573457 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:22.573418 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" event={"ID":"d200eb10-0d27-49e0-807e-37a41ee4a1cb","Type":"ContainerStarted","Data":"4e28e395aa3180a40efb9e3e06b94f2ad32ef15024d5ba502c80e8b6a1403f95"} Apr 16 08:46:22.573881 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:22.573620 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:46:22.575286 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:22.575245 2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-55ff986f96-bbtrf container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Apr 16 08:46:22.575410 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:22.575306 2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" podUID="d200eb10-0d27-49e0-807e-37a41ee4a1cb" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Apr 16 08:46:22.591720 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:22.591654 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" podStartSLOduration=1.8210659649999998 podStartE2EDuration="5.59163757s" podCreationTimestamp="2026-04-16 08:46:17 +0000 UTC" firstStartedPulling="2026-04-16 08:46:17.893583158 +0000 UTC m=+441.255595066" lastFinishedPulling="2026-04-16 08:46:21.664154763 +0000 UTC m=+445.026166671" observedRunningTime="2026-04-16 08:46:22.59055852 +0000 UTC m=+445.952570453" watchObservedRunningTime="2026-04-16 08:46:22.59163757 +0000 UTC m=+445.953649505" Apr 16 08:46:23.577794 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:46:23.577769 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-55ff986f96-bbtrf" Apr 16 08:47:12.704689 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.704652 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz"] Apr 16 08:47:12.711135 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.711107 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:12.713646 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.713619 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Apr 16 08:47:12.713787 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.713623 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Apr 16 08:47:12.714077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.714040 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-jgpck\"" Apr 16 08:47:12.720953 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.720931 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz"] Apr 16 08:47:12.869738 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.869685 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cdsh\" (UniqueName: \"kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh\") pod \"limitador-operator-controller-manager-85c4996f8c-szvmz\" (UID: \"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:12.970437 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.970347 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8cdsh\" (UniqueName: \"kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh\") pod \"limitador-operator-controller-manager-85c4996f8c-szvmz\" (UID: \"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:12.981653 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:12.981618 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cdsh\" (UniqueName: \"kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh\") pod \"limitador-operator-controller-manager-85c4996f8c-szvmz\" (UID: \"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:13.022692 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:13.022666 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:13.155389 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:13.155366 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz"] Apr 16 08:47:13.157832 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:47:13.157805 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da9cd4b_0fd1_4557_b886_f6c2df3a8c34.slice/crio-75bfdbfdf0e4951fa8cf7d21816931ff20f401b4ff694d8c0e7c267639a97619 WatchSource:0}: Error finding container 75bfdbfdf0e4951fa8cf7d21816931ff20f401b4ff694d8c0e7c267639a97619: Status 404 returned error can't find the container with id 75bfdbfdf0e4951fa8cf7d21816931ff20f401b4ff694d8c0e7c267639a97619 Apr 16 08:47:13.740539 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:13.740496 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" event={"ID":"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34","Type":"ContainerStarted","Data":"75bfdbfdf0e4951fa8cf7d21816931ff20f401b4ff694d8c0e7c267639a97619"} Apr 16 08:47:15.749507 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:15.749470 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" event={"ID":"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34","Type":"ContainerStarted","Data":"7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45"} Apr 16 08:47:15.749962 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:15.749555 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:15.763840 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:15.763667 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" podStartSLOduration=1.672738047 podStartE2EDuration="3.763651987s" podCreationTimestamp="2026-04-16 08:47:12 +0000 UTC" firstStartedPulling="2026-04-16 08:47:13.159763234 +0000 UTC m=+496.521775143" lastFinishedPulling="2026-04-16 08:47:15.25067716 +0000 UTC m=+498.612689083" observedRunningTime="2026-04-16 08:47:15.762914676 +0000 UTC m=+499.124926617" watchObservedRunningTime="2026-04-16 08:47:15.763651987 +0000 UTC m=+499.125663924" Apr 16 08:47:26.756030 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:26.755995 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:29.305506 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.305473 2569 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz"] Apr 16 08:47:29.305892 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.305727 2569 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" containerName="manager" containerID="cri-o://7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45" gracePeriod=2 Apr 16 08:47:29.312531 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.312465 2569 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz"] Apr 16 08:47:29.334614 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.334579 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6"] Apr 16 08:47:29.334987 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.334971 2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" containerName="manager" Apr 16 08:47:29.335033 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.334991 2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" containerName="manager" Apr 16 08:47:29.335093 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.335087 2569 memory_manager.go:356] "RemoveStaleState removing state" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" containerName="manager" Apr 16 08:47:29.338207 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.338184 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:29.339916 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.339881 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:29.346778 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.346749 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6"] Apr 16 08:47:29.504663 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.504635 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7884\" (UniqueName: \"kubernetes.io/projected/b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5-kube-api-access-d7884\") pod \"limitador-operator-controller-manager-85c4996f8c-lnth6\" (UID: \"b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:29.540703 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.540681 2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:29.542877 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.542851 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:29.605773 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.605706 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d7884\" (UniqueName: \"kubernetes.io/projected/b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5-kube-api-access-d7884\") pod \"limitador-operator-controller-manager-85c4996f8c-lnth6\" (UID: \"b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:29.615549 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.615521 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7884\" (UniqueName: \"kubernetes.io/projected/b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5-kube-api-access-d7884\") pod \"limitador-operator-controller-manager-85c4996f8c-lnth6\" (UID: \"b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:29.694964 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.694934 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:29.706903 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.706878 2569 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8cdsh\" (UniqueName: \"kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh\") pod \"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34\" (UID: \"2da9cd4b-0fd1-4557-b886-f6c2df3a8c34\") " Apr 16 08:47:29.709104 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.709081 2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh" (OuterVolumeSpecName: "kube-api-access-8cdsh") pod "2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" (UID: "2da9cd4b-0fd1-4557-b886-f6c2df3a8c34"). InnerVolumeSpecName "kube-api-access-8cdsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Apr 16 08:47:29.796653 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.796617 2569 generic.go:358] "Generic (PLEG): container finished" podID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" containerID="7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45" exitCode=0 Apr 16 08:47:29.796791 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.796667 2569 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" Apr 16 08:47:29.796791 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.796711 2569 scope.go:117] "RemoveContainer" containerID="7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45" Apr 16 08:47:29.798930 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.798904 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:29.805077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.805040 2569 scope.go:117] "RemoveContainer" containerID="7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45" Apr 16 08:47:29.805362 ip-10-0-137-248 kubenswrapper[2569]: E0416 08:47:29.805343 2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45\": container with ID starting with 7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45 not found: ID does not exist" containerID="7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45" Apr 16 08:47:29.805408 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.805371 2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45"} err="failed to get container status \"7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45\": rpc error: code = NotFound desc = could not find container \"7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45\": container with ID starting with 7451af04c2caa2559a71f6f5ef097effd2bfbbaf1f453010c98c487dcae8fe45 not found: ID does not exist" Apr 16 08:47:29.806841 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.806818 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:29.808082 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.808044 2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8cdsh\" (UniqueName: \"kubernetes.io/projected/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34-kube-api-access-8cdsh\") on node \"ip-10-0-137-248.ec2.internal\" DevicePath \"\"" Apr 16 08:47:29.820156 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:29.820126 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6"] Apr 16 08:47:29.823114 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:47:29.823091 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d2e4ec_9ffb_414b_8de8_5f45d47fb5f5.slice/crio-baecb5d5153d9f8035692bfdd2c29e95c6cdbe772d73d8e886a02128a27b2661 WatchSource:0}: Error finding container baecb5d5153d9f8035692bfdd2c29e95c6cdbe772d73d8e886a02128a27b2661: Status 404 returned error can't find the container with id baecb5d5153d9f8035692bfdd2c29e95c6cdbe772d73d8e886a02128a27b2661 Apr 16 08:47:30.482907 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.482872 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p"] Apr 16 08:47:30.485978 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.485955 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.488882 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.488863 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zv6tx\"" Apr 16 08:47:30.497816 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.497796 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p"] Apr 16 08:47:30.522039 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.522007 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:30.613404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.613331 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bpld\" (UniqueName: \"kubernetes.io/projected/cd324fdb-eb11-4a9d-9517-67d826ae1309-kube-api-access-5bpld\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.613404 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.613378 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd324fdb-eb11-4a9d-9517-67d826ae1309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.713741 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.713709 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5bpld\" (UniqueName: \"kubernetes.io/projected/cd324fdb-eb11-4a9d-9517-67d826ae1309-kube-api-access-5bpld\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.713881 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.713762 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd324fdb-eb11-4a9d-9517-67d826ae1309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.714162 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.714145 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd324fdb-eb11-4a9d-9517-67d826ae1309-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.721555 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.721520 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bpld\" (UniqueName: \"kubernetes.io/projected/cd324fdb-eb11-4a9d-9517-67d826ae1309-kube-api-access-5bpld\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p\" (UID: \"cd324fdb-eb11-4a9d-9517-67d826ae1309\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.795408 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.795363 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:30.801101 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.801043 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" event={"ID":"b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5","Type":"ContainerStarted","Data":"2686da264cd7914520e860f33df40352d3ae8613c58a6ed4e198562459c6f4e3"} Apr 16 08:47:30.801218 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.801105 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" event={"ID":"b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5","Type":"ContainerStarted","Data":"baecb5d5153d9f8035692bfdd2c29e95c6cdbe772d73d8e886a02128a27b2661"} Apr 16 08:47:30.801218 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.801154 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:30.816678 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.816629 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" podStartSLOduration=1.816613056 podStartE2EDuration="1.816613056s" podCreationTimestamp="2026-04-16 08:47:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 08:47:30.816431281 +0000 UTC m=+514.178443210" watchObservedRunningTime="2026-04-16 08:47:30.816613056 +0000 UTC m=+514.178624998" Apr 16 08:47:30.817853 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.817824 2569 status_manager.go:895] "Failed to get status for pod" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-szvmz" err="pods \"limitador-operator-controller-manager-85c4996f8c-szvmz\" is forbidden: User \"system:node:ip-10-0-137-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-248.ec2.internal' and this object" Apr 16 08:47:30.923360 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:30.923335 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p"] Apr 16 08:47:30.926317 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:47:30.926286 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd324fdb_eb11_4a9d_9517_67d826ae1309.slice/crio-96cb86958ef0ce45733885e98eec71b937fa4abf95ae4b2e9f362a379679d9f7 WatchSource:0}: Error finding container 96cb86958ef0ce45733885e98eec71b937fa4abf95ae4b2e9f362a379679d9f7: Status 404 returned error can't find the container with id 96cb86958ef0ce45733885e98eec71b937fa4abf95ae4b2e9f362a379679d9f7 Apr 16 08:47:31.257071 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:31.257029 2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2da9cd4b-0fd1-4557-b886-f6c2df3a8c34" path="/var/lib/kubelet/pods/2da9cd4b-0fd1-4557-b886-f6c2df3a8c34/volumes" Apr 16 08:47:31.807278 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:31.807233 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" event={"ID":"cd324fdb-eb11-4a9d-9517-67d826ae1309","Type":"ContainerStarted","Data":"96cb86958ef0ce45733885e98eec71b937fa4abf95ae4b2e9f362a379679d9f7"} Apr 16 08:47:34.818921 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:34.818880 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" event={"ID":"cd324fdb-eb11-4a9d-9517-67d826ae1309","Type":"ContainerStarted","Data":"f913c06fd0bceac5b30268b65064b6a302fe23be516baea690b4aabc2cfc198d"} Apr 16 08:47:34.819334 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:34.819040 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:47:34.839960 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:34.839916 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" podStartSLOduration=1.429423205 podStartE2EDuration="4.839902679s" podCreationTimestamp="2026-04-16 08:47:30 +0000 UTC" firstStartedPulling="2026-04-16 08:47:30.928784026 +0000 UTC m=+514.290795934" lastFinishedPulling="2026-04-16 08:47:34.3392635 +0000 UTC m=+517.701275408" observedRunningTime="2026-04-16 08:47:34.8382804 +0000 UTC m=+518.200292369" watchObservedRunningTime="2026-04-16 08:47:34.839902679 +0000 UTC m=+518.201914608" Apr 16 08:47:41.809740 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:41.809707 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-lnth6" Apr 16 08:47:45.825078 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:47:45.825017 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p" Apr 16 08:48:41.835535 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.835498 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5c4547b6f9-mnbfp"] Apr 16 08:48:41.838754 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.838731 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.840541 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.840508 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Apr 16 08:48:41.841107 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.841089 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-rxq2t\"" Apr 16 08:48:41.841172 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.841108 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Apr 16 08:48:41.848440 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.848419 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5c4547b6f9-mnbfp"] Apr 16 08:48:41.874369 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.874346 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a28751-7036-4e0f-8ce2-b0d18f83424d-maas-api-tls\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.874479 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.874384 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nm94\" (UniqueName: \"kubernetes.io/projected/f2a28751-7036-4e0f-8ce2-b0d18f83424d-kube-api-access-8nm94\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.975077 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.975027 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nm94\" (UniqueName: \"kubernetes.io/projected/f2a28751-7036-4e0f-8ce2-b0d18f83424d-kube-api-access-8nm94\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.975244 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.975139 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a28751-7036-4e0f-8ce2-b0d18f83424d-maas-api-tls\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.977698 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.977657 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2a28751-7036-4e0f-8ce2-b0d18f83424d-maas-api-tls\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:41.989433 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:41.989408 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nm94\" (UniqueName: \"kubernetes.io/projected/f2a28751-7036-4e0f-8ce2-b0d18f83424d-kube-api-access-8nm94\") pod \"maas-api-5c4547b6f9-mnbfp\" (UID: \"f2a28751-7036-4e0f-8ce2-b0d18f83424d\") " pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:42.149283 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:42.149257 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:42.272501 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:42.272320 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5c4547b6f9-mnbfp"] Apr 16 08:48:42.274918 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:48:42.274893 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a28751_7036_4e0f_8ce2_b0d18f83424d.slice/crio-4ea8b02921272a8f6338c3f1fc3fe89b2888c0d48254529570c9bfa3f2c6dfc7 WatchSource:0}: Error finding container 4ea8b02921272a8f6338c3f1fc3fe89b2888c0d48254529570c9bfa3f2c6dfc7: Status 404 returned error can't find the container with id 4ea8b02921272a8f6338c3f1fc3fe89b2888c0d48254529570c9bfa3f2c6dfc7 Apr 16 08:48:43.042033 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:43.041987 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" event={"ID":"f2a28751-7036-4e0f-8ce2-b0d18f83424d","Type":"ContainerStarted","Data":"4ea8b02921272a8f6338c3f1fc3fe89b2888c0d48254529570c9bfa3f2c6dfc7"} Apr 16 08:48:45.049720 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:45.049684 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" event={"ID":"f2a28751-7036-4e0f-8ce2-b0d18f83424d","Type":"ContainerStarted","Data":"0e4e330debc32db957c1ec5ca32be527968f8e6516f3d2d2dac1e82880964579"} Apr 16 08:48:45.050242 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:45.049800 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:45.063504 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:45.063453 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" podStartSLOduration=2.007707314 podStartE2EDuration="4.063404119s" podCreationTimestamp="2026-04-16 08:48:41 +0000 UTC" firstStartedPulling="2026-04-16 08:48:42.276828376 +0000 UTC m=+585.638840297" lastFinishedPulling="2026-04-16 08:48:44.332525191 +0000 UTC m=+587.694537102" observedRunningTime="2026-04-16 08:48:45.063235292 +0000 UTC m=+588.425247232" watchObservedRunningTime="2026-04-16 08:48:45.063404119 +0000 UTC m=+588.425416050" Apr 16 08:48:51.058132 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:51.058102 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5c4547b6f9-mnbfp" Apr 16 08:48:57.181258 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:57.181227 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:48:57.181671 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:48:57.181488 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:49:33.606540 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.606510 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95"] Apr 16 08:49:33.609920 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.609898 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.611726 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.611706 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-88shx\"" Apr 16 08:49:33.611811 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.611736 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Apr 16 08:49:33.612355 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.612336 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Apr 16 08:49:33.612411 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.612374 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Apr 16 08:49:33.617322 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.617114 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95"] Apr 16 08:49:33.693005 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.692977 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv688\" (UniqueName: \"kubernetes.io/projected/04b66d69-7990-4a16-8826-f0537b8aa016-kube-api-access-dv688\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.693168 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.693012 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b66d69-7990-4a16-8826-f0537b8aa016-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.693168 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.693064 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-dshm\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.693168 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.693131 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.693271 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.693170 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-home\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.693271 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.693204 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-model-cache\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.793825 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.793792 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-model-cache\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.793974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.793835 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dv688\" (UniqueName: \"kubernetes.io/projected/04b66d69-7990-4a16-8826-f0537b8aa016-kube-api-access-dv688\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.793974 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.793957 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b66d69-7990-4a16-8826-f0537b8aa016-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794045 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794032 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-dshm\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794116 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794087 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794168 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794127 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-home\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794313 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794291 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-model-cache\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794444 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794423 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.794507 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.794435 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-home\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.796380 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.796362 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04b66d69-7990-4a16-8826-f0537b8aa016-dshm\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.796636 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.796619 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04b66d69-7990-4a16-8826-f0537b8aa016-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.800547 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.800521 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv688\" (UniqueName: \"kubernetes.io/projected/04b66d69-7990-4a16-8826-f0537b8aa016-kube-api-access-dv688\") pod \"premium-simulated-simulated-premium-kserve-f5df4587b-wfz95\" (UID: \"04b66d69-7990-4a16-8826-f0537b8aa016\") " pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:33.919353 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:33.919322 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:34.047338 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:34.047313 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95"] Apr 16 08:49:34.049261 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:49:34.049235 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b66d69_7990_4a16_8826_f0537b8aa016.slice/crio-6d55fa8be6c50642069e3580f909c1a89126b00cc5ba314160b3ad77636a3e70 WatchSource:0}: Error finding container 6d55fa8be6c50642069e3580f909c1a89126b00cc5ba314160b3ad77636a3e70: Status 404 returned error can't find the container with id 6d55fa8be6c50642069e3580f909c1a89126b00cc5ba314160b3ad77636a3e70 Apr 16 08:49:34.209123 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:34.209026 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" event={"ID":"04b66d69-7990-4a16-8826-f0537b8aa016","Type":"ContainerStarted","Data":"6d55fa8be6c50642069e3580f909c1a89126b00cc5ba314160b3ad77636a3e70"} Apr 16 08:49:40.231598 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:40.231555 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" event={"ID":"04b66d69-7990-4a16-8826-f0537b8aa016","Type":"ContainerStarted","Data":"c093ecacb3ac50610768547e7add25370bd720ee015342d5c065573c1627adb2"} Apr 16 08:49:47.258491 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:47.258459 2569 generic.go:358] "Generic (PLEG): container finished" podID="04b66d69-7990-4a16-8826-f0537b8aa016" containerID="c093ecacb3ac50610768547e7add25370bd720ee015342d5c065573c1627adb2" exitCode=0 Apr 16 08:49:47.259427 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:47.259402 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" event={"ID":"04b66d69-7990-4a16-8826-f0537b8aa016","Type":"ContainerDied","Data":"c093ecacb3ac50610768547e7add25370bd720ee015342d5c065573c1627adb2"} Apr 16 08:49:51.280603 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:51.280570 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" event={"ID":"04b66d69-7990-4a16-8826-f0537b8aa016","Type":"ContainerStarted","Data":"333076d4f17bab119998c681fec57092ebc16d9d14143e53628312c584d897de"} Apr 16 08:49:51.280978 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:51.280766 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:49:51.296498 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:49:51.296451 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" podStartSLOduration=1.209520143 podStartE2EDuration="18.296424176s" podCreationTimestamp="2026-04-16 08:49:33 +0000 UTC" firstStartedPulling="2026-04-16 08:49:34.051347778 +0000 UTC m=+637.413359692" lastFinishedPulling="2026-04-16 08:49:51.138251803 +0000 UTC m=+654.500263725" observedRunningTime="2026-04-16 08:49:51.295657556 +0000 UTC m=+654.657669520" watchObservedRunningTime="2026-04-16 08:49:51.296424176 +0000 UTC m=+654.658436122" Apr 16 08:50:02.296860 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:02.296821 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-f5df4587b-wfz95" Apr 16 08:50:04.594593 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.594553 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6"] Apr 16 08:50:04.670368 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.670328 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6"] Apr 16 08:50:04.670540 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.670473 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.672739 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.672687 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Apr 16 08:50:04.757783 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.757743 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45074fce-2b95-49ce-941a-3da46839b86c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.757961 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.757806 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-home\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.757961 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.757891 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wck76\" (UniqueName: \"kubernetes.io/projected/45074fce-2b95-49ce-941a-3da46839b86c-kube-api-access-wck76\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.757961 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.757943 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.758116 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.758004 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.758116 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.758064 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859240 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859139 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45074fce-2b95-49ce-941a-3da46839b86c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859240 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859212 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-home\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859472 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859247 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wck76\" (UniqueName: \"kubernetes.io/projected/45074fce-2b95-49ce-941a-3da46839b86c-kube-api-access-wck76\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859472 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859285 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859472 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859331 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859472 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859368 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859703 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859682 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-home\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859703 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859696 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.859914 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.859891 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.862042 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.862013 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/45074fce-2b95-49ce-941a-3da46839b86c-dshm\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.862314 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.862295 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45074fce-2b95-49ce-941a-3da46839b86c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.866362 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.866343 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wck76\" (UniqueName: \"kubernetes.io/projected/45074fce-2b95-49ce-941a-3da46839b86c-kube-api-access-wck76\") pod \"e2e-distinct-2-simulated-kserve-8454f99c75-28qx6\" (UID: \"45074fce-2b95-49ce-941a-3da46839b86c\") " pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:04.981241 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:04.981205 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:05.110828 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:05.110804 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6"] Apr 16 08:50:05.113222 ip-10-0-137-248 kubenswrapper[2569]: W0416 08:50:05.113195 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45074fce_2b95_49ce_941a_3da46839b86c.slice/crio-56f7f7a034e55715d7100d720ea18fb753eb5685394ec3c6edaf0453d23b61f2 WatchSource:0}: Error finding container 56f7f7a034e55715d7100d720ea18fb753eb5685394ec3c6edaf0453d23b61f2: Status 404 returned error can't find the container with id 56f7f7a034e55715d7100d720ea18fb753eb5685394ec3c6edaf0453d23b61f2 Apr 16 08:50:05.115140 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:05.115123 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 08:50:05.326315 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:05.326281 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" event={"ID":"45074fce-2b95-49ce-941a-3da46839b86c","Type":"ContainerStarted","Data":"9197360d50fd584de61ae9ab5fd3ecc52ae18b98563823b506d9204990175fb7"} Apr 16 08:50:05.326466 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:05.326323 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" event={"ID":"45074fce-2b95-49ce-941a-3da46839b86c","Type":"ContainerStarted","Data":"56f7f7a034e55715d7100d720ea18fb753eb5685394ec3c6edaf0453d23b61f2"} Apr 16 08:50:11.346512 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:11.346470 2569 generic.go:358] "Generic (PLEG): container finished" podID="45074fce-2b95-49ce-941a-3da46839b86c" containerID="9197360d50fd584de61ae9ab5fd3ecc52ae18b98563823b506d9204990175fb7" exitCode=0 Apr 16 08:50:11.346966 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:11.346543 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" event={"ID":"45074fce-2b95-49ce-941a-3da46839b86c","Type":"ContainerDied","Data":"9197360d50fd584de61ae9ab5fd3ecc52ae18b98563823b506d9204990175fb7"} Apr 16 08:50:12.351597 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:12.351555 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" event={"ID":"45074fce-2b95-49ce-941a-3da46839b86c","Type":"ContainerStarted","Data":"71fc88349583210071c23180e1ccb9a6d2ca4fa7482c4098e99c5e08053ffc9e"} Apr 16 08:50:12.352112 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:12.351794 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:50:12.368207 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:12.368151 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" podStartSLOduration=8.141580741 podStartE2EDuration="8.368136719s" podCreationTimestamp="2026-04-16 08:50:04 +0000 UTC" firstStartedPulling="2026-04-16 08:50:11.347153832 +0000 UTC m=+674.709165741" lastFinishedPulling="2026-04-16 08:50:11.573709797 +0000 UTC m=+674.935721719" observedRunningTime="2026-04-16 08:50:12.367170204 +0000 UTC m=+675.729182134" watchObservedRunningTime="2026-04-16 08:50:12.368136719 +0000 UTC m=+675.730148651" Apr 16 08:50:23.368850 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:50:23.368824 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8454f99c75-28qx6" Apr 16 08:53:57.202668 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:53:57.202573 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:53:57.203269 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:53:57.203248 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:58:57.223485 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:58:57.223459 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 08:58:57.225913 ip-10-0-137-248 kubenswrapper[2569]: I0416 08:58:57.224983 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:03:57.247859 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:03:57.247746 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:03:57.250996 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:03:57.249898 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:08:38.534555 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:38.534476 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-qhwqx_deee6c74-ed99-49c0-86e0-ad7cd7fe230f/manager/0.log" Apr 16 09:08:38.654621 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:38.654587 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5c4547b6f9-mnbfp_f2a28751-7036-4e0f-8ce2-b0d18f83424d/maas-api/0.log" Apr 16 09:08:38.896649 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:38.896564 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7ffb6_fc3c99a7-e248-47f6-9ef2-b554a48c4a8d/manager/1.log" Apr 16 09:08:39.151717 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:39.151679 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-569944d57d-wlnj8_f3286354-03df-4614-990c-249fbe6bd7f3/manager/0.log" Apr 16 09:08:41.216409 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:41.216375 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p_cd324fdb-eb11-4a9d-9517-67d826ae1309/manager/0.log" Apr 16 09:08:41.443843 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:41.443808 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-lnth6_b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5/manager/0.log" Apr 16 09:08:41.910181 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:41.910144 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-bbtrf_d200eb10-0d27-49e0-807e-37a41ee4a1cb/discovery/0.log" Apr 16 09:08:42.020731 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:42.020698 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d5746998b-cxkd7_e0a114ba-6f14-49f3-b21e-394243bb6f59/kube-auth-proxy/0.log" Apr 16 09:08:42.704851 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:42.704814 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8454f99c75-28qx6_45074fce-2b95-49ce-941a-3da46839b86c/storage-initializer/0.log" Apr 16 09:08:42.712460 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:42.712436 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8454f99c75-28qx6_45074fce-2b95-49ce-941a-3da46839b86c/main/0.log" Apr 16 09:08:43.187981 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:43.187948 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-f5df4587b-wfz95_04b66d69-7990-4a16-8826-f0537b8aa016/storage-initializer/0.log" Apr 16 09:08:43.195897 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:43.195862 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-f5df4587b-wfz95_04b66d69-7990-4a16-8826-f0537b8aa016/main/0.log" Apr 16 09:08:50.275573 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:50.275535 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4vp8m_1597cea1-f09d-46f2-9513-72dde7873326/global-pull-secret-syncer/0.log" Apr 16 09:08:50.428063 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:50.428008 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8zxqc_b10f80d7-ad09-4e3c-b75a-ec48f22e39be/konnectivity-agent/0.log" Apr 16 09:08:50.486496 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:50.486464 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-248.ec2.internal_7a9f331420ac4d3ee0faecb39c870566/haproxy/0.log" Apr 16 09:08:54.818020 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:54.817985 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-6bc9f4c76f-9jf4p_cd324fdb-eb11-4a9d-9517-67d826ae1309/manager/0.log" Apr 16 09:08:54.967878 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:54.967841 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-lnth6_b5d2e4ec-9ffb-414b-8de8-5f45d47fb5f5/manager/0.log" Apr 16 09:08:56.799023 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:56.798991 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srh26_723d8931-2012-4f4b-91ed-79a2994a8f92/node-exporter/0.log" Apr 16 09:08:56.819147 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:56.819119 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srh26_723d8931-2012-4f4b-91ed-79a2994a8f92/kube-rbac-proxy/0.log" Apr 16 09:08:56.839820 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:56.839798 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srh26_723d8931-2012-4f4b-91ed-79a2994a8f92/init-textfile/0.log" Apr 16 09:08:57.270583 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:57.270486 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:08:57.280335 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:57.274278 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:08:58.725220 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.725184 2569 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh"] Apr 16 09:08:58.728377 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.728359 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.730469 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.730448 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmdlf\"/\"openshift-service-ca.crt\"" Apr 16 09:08:58.731073 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.731033 2569 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mmdlf\"/\"kube-root-ca.crt\"" Apr 16 09:08:58.731161 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.731039 2569 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mmdlf\"/\"default-dockercfg-9mlmk\"" Apr 16 09:08:58.738935 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.738907 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh"] Apr 16 09:08:58.888162 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.888115 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-lib-modules\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.888162 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.888152 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj5vd\" (UniqueName: \"kubernetes.io/projected/ca476354-5700-4a6f-b193-10efdac90910-kube-api-access-hj5vd\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.888388 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.888186 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-proc\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.888388 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.888260 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-sys\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.888388 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.888339 2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-podres\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.988934 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988841 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-sys\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.988934 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988906 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-podres\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.988934 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988937 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-lib-modules\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988955 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hj5vd\" (UniqueName: \"kubernetes.io/projected/ca476354-5700-4a6f-b193-10efdac90910-kube-api-access-hj5vd\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988973 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-sys\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.988987 2569 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-proc\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.989042 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-proc\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.989090 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-podres\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.989231 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.989125 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ca476354-5700-4a6f-b193-10efdac90910-lib-modules\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:58.997378 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:58.997358 2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj5vd\" (UniqueName: \"kubernetes.io/projected/ca476354-5700-4a6f-b193-10efdac90910-kube-api-access-hj5vd\") pod \"perf-node-gather-daemonset-xdrgh\" (UID: \"ca476354-5700-4a6f-b193-10efdac90910\") " pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:59.039379 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:59.039346 2569 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:08:59.166296 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:59.166263 2569 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh"] Apr 16 09:08:59.169010 ip-10-0-137-248 kubenswrapper[2569]: W0416 09:08:59.168979 2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podca476354_5700_4a6f_b193_10efdac90910.slice/crio-59f4de850b041c4cae5888439d3913945e9c899ac67229d29598a5a13b626594 WatchSource:0}: Error finding container 59f4de850b041c4cae5888439d3913945e9c899ac67229d29598a5a13b626594: Status 404 returned error can't find the container with id 59f4de850b041c4cae5888439d3913945e9c899ac67229d29598a5a13b626594 Apr 16 09:08:59.170638 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:08:59.170623 2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Apr 16 09:09:00.172503 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.172464 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" event={"ID":"ca476354-5700-4a6f-b193-10efdac90910","Type":"ContainerStarted","Data":"651f70d401d0af174d321e59c8f3c598836afca51cd824047b30093660389c55"} Apr 16 09:09:00.172503 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.172506 2569 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" event={"ID":"ca476354-5700-4a6f-b193-10efdac90910","Type":"ContainerStarted","Data":"59f4de850b041c4cae5888439d3913945e9c899ac67229d29598a5a13b626594"} Apr 16 09:09:00.172940 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.172593 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:09:00.186399 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.186357 2569 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" podStartSLOduration=2.186343857 podStartE2EDuration="2.186343857s" podCreationTimestamp="2026-04-16 09:08:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-04-16 09:09:00.184898758 +0000 UTC m=+1803.546910688" watchObservedRunningTime="2026-04-16 09:09:00.186343857 +0000 UTC m=+1803.548355822" Apr 16 09:09:00.851697 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.851657 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7fxz_48959348-ef25-4f02-8521-38b7e30ce7a1/dns/0.log" Apr 16 09:09:00.870307 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.870281 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-h7fxz_48959348-ef25-4f02-8521-38b7e30ce7a1/kube-rbac-proxy/0.log" Apr 16 09:09:00.964654 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:00.964623 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k2pss_0ee13c74-b0d0-42a3-959b-963a8f04c0b0/dns-node-resolver/0.log" Apr 16 09:09:01.465664 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:01.465617 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7gcs5_5a824f8c-b47c-4092-8fa7-c2564991fd54/node-ca/0.log" Apr 16 09:09:02.453536 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:02.453497 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-55ff986f96-bbtrf_d200eb10-0d27-49e0-807e-37a41ee4a1cb/discovery/0.log" Apr 16 09:09:02.475938 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:02.475900 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d5746998b-cxkd7_e0a114ba-6f14-49f3-b21e-394243bb6f59/kube-auth-proxy/0.log" Apr 16 09:09:03.115253 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:03.115224 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ps6df_c63ff564-0d80-4740-8ca8-39f1950fe05a/serve-healthcheck-canary/0.log" Apr 16 09:09:03.576370 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:03.576339 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22kr6_516340c2-5ce8-4e39-8636-3b98afc987b1/kube-rbac-proxy/0.log" Apr 16 09:09:03.597204 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:03.597167 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22kr6_516340c2-5ce8-4e39-8636-3b98afc987b1/exporter/0.log" Apr 16 09:09:03.618519 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:03.618483 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-22kr6_516340c2-5ce8-4e39-8636-3b98afc987b1/extractor/0.log" Apr 16 09:09:05.573044 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:05.573008 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-qhwqx_deee6c74-ed99-49c0-86e0-ad7cd7fe230f/manager/0.log" Apr 16 09:09:05.614192 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:05.614158 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5c4547b6f9-mnbfp_f2a28751-7036-4e0f-8ce2-b0d18f83424d/maas-api/0.log" Apr 16 09:09:05.709496 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:05.709466 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7ffb6_fc3c99a7-e248-47f6-9ef2-b554a48c4a8d/manager/0.log" Apr 16 09:09:05.730190 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:05.730157 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7ffb6_fc3c99a7-e248-47f6-9ef2-b554a48c4a8d/manager/1.log" Apr 16 09:09:05.776923 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:05.776892 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-569944d57d-wlnj8_f3286354-03df-4614-990c-249fbe6bd7f3/manager/0.log" Apr 16 09:09:06.186382 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:06.186356 2569 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mmdlf/perf-node-gather-daemonset-xdrgh" Apr 16 09:09:13.049733 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.049708 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/kube-multus-additional-cni-plugins/0.log" Apr 16 09:09:13.071176 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.071152 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/egress-router-binary-copy/0.log" Apr 16 09:09:13.091999 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.091962 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/cni-plugins/0.log" Apr 16 09:09:13.116033 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.116007 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/bond-cni-plugin/0.log" Apr 16 09:09:13.138630 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.138603 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/routeoverride-cni/0.log" Apr 16 09:09:13.163796 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.163758 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/whereabouts-cni-bincopy/0.log" Apr 16 09:09:13.187218 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.187182 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rzw5p_3759b22a-834b-4461-859a-8702206c5593/whereabouts-cni/0.log" Apr 16 09:09:13.247807 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.247762 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fjk5h_80259a0b-bf2a-4d3c-886c-62ed1c5b20b6/kube-multus/0.log" Apr 16 09:09:13.401612 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.401571 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mcq96_1b29b84e-b6c8-46e1-891b-df2585f55350/network-metrics-daemon/0.log" Apr 16 09:09:13.421860 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:13.421835 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mcq96_1b29b84e-b6c8-46e1-891b-df2585f55350/kube-rbac-proxy/0.log" Apr 16 09:09:14.880764 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.880736 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-controller/0.log" Apr 16 09:09:14.898133 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.898108 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/0.log" Apr 16 09:09:14.913326 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.913296 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovn-acl-logging/1.log" Apr 16 09:09:14.934298 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.934265 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/kube-rbac-proxy-node/0.log" Apr 16 09:09:14.955821 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.955791 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/kube-rbac-proxy-ovn-metrics/0.log" Apr 16 09:09:14.974740 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.974710 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/northd/0.log" Apr 16 09:09:14.997472 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:14.997445 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/nbdb/0.log" Apr 16 09:09:15.018186 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:15.018159 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/sbdb/0.log" Apr 16 09:09:15.185990 ip-10-0-137-248 kubenswrapper[2569]: I0416 09:09:15.185960 2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xkwdc_af3932be-73ab-4191-8a83-b4ead0398830/ovnkube-controller/0.log"