Jun 02 21:39:32.061307 ip-10-0-143-61 systemd[1]: Starting Kubernetes Kubelet... Jun 02 21:39:32.566191 ip-10-0-143-61 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:39:32.566191 ip-10-0-143-61 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 21:39:32.566191 ip-10-0-143-61 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:39:32.566191 ip-10-0-143-61 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 21:39:32.566191 ip-10-0-143-61 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:39:32.569235 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.569082 2558 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 21:39:32.573427 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573410 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:39:32.573427 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573427 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573433 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573436 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573439 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573442 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573445 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573448 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573451 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573454 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573456 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573459 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573462 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573466 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573469 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573472 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573475 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573479 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573483 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573486 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:39:32.573499 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573489 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573491 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573494 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573497 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573499 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573502 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573505 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573508 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573511 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573513 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573516 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573519 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573521 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573524 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573527 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573529 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573532 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573534 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573538 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573542 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:39:32.573941 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573544 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573547 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573549 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573552 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573554 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573557 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573560 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573563 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573567 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573569 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573572 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573574 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573577 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573580 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573583 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573586 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573588 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573591 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573593 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573596 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:39:32.574426 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573598 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573601 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573604 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573606 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573609 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573612 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573615 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573618 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573620 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573622 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573625 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573628 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573631 2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573634 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573636 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573640 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573642 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573645 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573647 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573650 2558 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:39:32.574903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573653 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:39:32.575386 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573656 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:39:32.575386 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573658 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:39:32.575386 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573661 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:39:32.575386 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573664 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:39:32.575386 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.573666 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:39:32.576306 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576293 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:39:32.576306 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576305 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576308 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576311 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576314 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576317 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576319 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576322 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576325 2558 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576327 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576330 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576333 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576336 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576338 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576341 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576344 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576347 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576350 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576352 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576355 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576357 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:39:32.576366 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576360 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576363 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576365 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576368 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576372 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576374 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576377 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576380 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576383 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576385 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576388 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576390 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576393 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576395 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576398 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576400 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576402 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576405 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576407 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576410 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:39:32.576893 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576412 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576415 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576417 2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576420 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576422 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576425 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576427 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576429 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576431 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576435 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576437 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576441 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576444 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576447 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576451 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576454 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576457 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576460 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576463 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576465 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:39:32.577402 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576468 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576471 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576473 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576476 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576479 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576481 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576484 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576486 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576489 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576491 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576494 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576496 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576498 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576501 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576503 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576506 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576509 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576512 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576515 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:39:32.577891 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576519 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576522 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576524 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576527 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576529 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.576532 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576604 2558 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576613 2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576620 2558 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576625 2558 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576630 2558 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576634 2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576639 2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576644 2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576647 2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576650 2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576654 2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576657 2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576660 2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576663 2558 flags.go:64] FLAG: --cgroup-root="" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576666 2558 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576669 2558 flags.go:64] FLAG: --client-ca-file="" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576672 2558 flags.go:64] FLAG: --cloud-config="" Jun 02 21:39:32.578360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576675 2558 flags.go:64] FLAG: --cloud-provider="external" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576678 2558 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576681 2558 flags.go:64] FLAG: --cluster-domain="" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576684 2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576687 2558 flags.go:64] FLAG: --config-dir="" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576690 2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576693 2558 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576697 2558 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576701 2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576704 2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576707 2558 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576710 2558 flags.go:64] FLAG: --contention-profiling="false" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576713 2558 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576716 2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576720 2558 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576723 2558 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576727 2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576730 2558 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576733 2558 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576736 2558 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576740 2558 flags.go:64] FLAG: --enable-server="true" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576750 2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576755 2558 flags.go:64] FLAG: --event-burst="100" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576758 2558 flags.go:64] FLAG: --event-qps="50" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576761 2558 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 21:39:32.578911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576765 2558 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576768 2558 flags.go:64] FLAG: --eviction-hard="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576776 2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576779 2558 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576782 2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576785 2558 flags.go:64] FLAG: --eviction-soft="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576788 2558 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576791 2558 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576794 2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576797 2558 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576800 2558 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576804 2558 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576807 2558 flags.go:64] FLAG: --feature-gates="" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576811 2558 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576814 2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576818 2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576822 2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576825 2558 flags.go:64] FLAG: --healthz-port="10248" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576828 2558 flags.go:64] FLAG: --help="false" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576831 2558 flags.go:64] FLAG: --hostname-override="ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576835 2558 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576838 2558 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576841 2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576845 2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 21:39:32.579533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576849 2558 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576852 2558 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576855 2558 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576858 2558 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576861 2558 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576865 2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576868 2558 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576871 2558 flags.go:64] FLAG: --kube-reserved="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576883 2558 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576886 2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576889 2558 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576892 2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576895 2558 flags.go:64] FLAG: --lock-file="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576898 2558 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576901 2558 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576904 2558 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576910 2558 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576918 2558 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576921 2558 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576923 2558 flags.go:64] FLAG: --logging-format="text" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576926 2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576930 2558 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576933 2558 flags.go:64] FLAG: --manifest-url="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576936 2558 flags.go:64] FLAG: --manifest-url-header="" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576941 2558 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 21:39:32.580131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576945 2558 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576949 2558 flags.go:64] FLAG: --max-pods="110" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576952 2558 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576955 2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576958 2558 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576961 2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576964 2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576967 2558 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576970 2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576978 2558 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576981 2558 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.576984 2558 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577001 2558 flags.go:64] FLAG: --pod-cidr="" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577004 2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577010 2558 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577013 2558 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577017 2558 flags.go:64] FLAG: --pods-per-core="0" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577020 2558 flags.go:64] FLAG: --port="10250" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577023 2558 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577026 2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-029dd13a81e1aac27" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577029 2558 flags.go:64] FLAG: --qos-reserved="" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577032 2558 flags.go:64] FLAG: --read-only-port="10255" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577035 2558 flags.go:64] FLAG: --register-node="true" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577038 2558 flags.go:64] FLAG: --register-schedulable="true" Jun 02 21:39:32.580713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577041 2558 flags.go:64] FLAG: --register-with-taints="" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577046 2558 flags.go:64] FLAG: --registry-burst="10" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577050 2558 flags.go:64] FLAG: --registry-qps="5" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577053 2558 flags.go:64] FLAG: --reserved-cpus="" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577056 2558 flags.go:64] FLAG: --reserved-memory="" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577061 2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577064 2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577068 2558 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577070 2558 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577073 2558 flags.go:64] FLAG: --runonce="false" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577076 2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577080 2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577083 2558 flags.go:64] FLAG: --seccomp-default="false" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577085 2558 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577088 2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577092 2558 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577095 2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577098 2558 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577101 2558 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577103 2558 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577107 2558 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577109 2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577113 2558 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577116 2558 flags.go:64] FLAG: --system-cgroups="" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577119 2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 21:39:32.581301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577124 2558 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577127 2558 flags.go:64] FLAG: --tls-cert-file="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577130 2558 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577134 2558 flags.go:64] FLAG: --tls-min-version="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577136 2558 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577139 2558 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577142 2558 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577145 2558 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577150 2558 flags.go:64] FLAG: --v="2" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577154 2558 flags.go:64] FLAG: --version="false" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577159 2558 flags.go:64] FLAG: --vmodule="" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577163 2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.577167 2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577266 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577270 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577273 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577276 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577278 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577281 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577284 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577286 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577289 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577292 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:39:32.581950 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577295 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577298 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577300 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577303 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577305 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577310 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577313 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577318 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577322 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577325 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577328 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577331 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577336 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577339 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577341 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577344 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577347 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577351 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577354 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:39:32.582541 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577356 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577359 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577361 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577364 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577366 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577369 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577371 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577374 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577376 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577379 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577381 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577384 2558 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577387 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577389 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577391 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577394 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577397 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577399 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577402 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577404 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:39:32.583053 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577407 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577410 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577413 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577416 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577419 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577426 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577429 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577431 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577434 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577436 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577439 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577442 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577444 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577447 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577450 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577452 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577454 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577457 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577459 2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577462 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:39:32.583548 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577464 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577467 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577469 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577472 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577474 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577476 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577479 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577481 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577484 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577487 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577489 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577492 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577494 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577497 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577500 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577502 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:39:32.584208 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.577505 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:39:32.584912 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.578311 2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:39:32.586764 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.586742 2558 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 21:39:32.586764 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.586762 2558 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586814 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586820 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586824 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586827 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586830 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586832 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586835 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586837 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586841 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586843 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586847 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586852 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586854 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586857 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586860 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586862 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586865 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586868 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586871 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:39:32.586897 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586873 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586876 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586878 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586881 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586884 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586886 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586890 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586893 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586897 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586902 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586906 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586909 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586913 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586916 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586919 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586922 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586925 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586928 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586931 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:39:32.587412 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586933 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586936 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586939 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586942 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586944 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586946 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586949 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586952 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586954 2558 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586957 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586959 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586962 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586965 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586968 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586970 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586974 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586977 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586979 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.586982 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587002 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:39:32.587911 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587007 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587011 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587015 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587019 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587022 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587025 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587028 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587030 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587033 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587035 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587038 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587040 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587043 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587046 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587048 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587051 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587053 2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587056 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587059 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587062 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:39:32.588485 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587065 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587068 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587070 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587073 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587075 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587078 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587080 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587084 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.587089 2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587196 2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587201 2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587205 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587208 2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587210 2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587213 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:39:32.588978 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587216 2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587219 2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587221 2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587224 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587226 2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587229 2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587231 2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587234 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587236 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587240 2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587242 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587245 2558 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587247 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587250 2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587253 2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587255 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587260 2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587263 2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587266 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587268 2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:39:32.589372 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587271 2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587273 2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587276 2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587278 2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587281 2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587284 2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587287 2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587290 2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587292 2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587295 2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587298 2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587301 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587303 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587306 2558 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587309 2558 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587311 2558 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587314 2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587316 2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587319 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587321 2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:39:32.589856 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587324 2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587326 2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587329 2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587331 2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587334 2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587336 2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587339 2558 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587341 2558 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587344 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587347 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587350 2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587353 2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587356 2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587358 2558 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587361 2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587364 2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587367 2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587370 2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587372 2558 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587375 2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:39:32.590367 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587378 2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587380 2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587383 2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587385 2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587388 2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587391 2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587393 2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587396 2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587398 2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587401 2558 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587405 2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587408 2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587412 2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587416 2558 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587419 2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587422 2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587425 2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587428 2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587431 2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:39:32.590847 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:32.587434 2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:39:32.591372 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.587439 2558 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:39:32.591372 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.588324 2558 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 21:39:32.595524 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.595507 2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 21:39:32.596686 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.596668 2558 server.go:1019] "Starting client certificate rotation" Jun 02 21:39:32.596809 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.596790 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:39:32.596847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.596838 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:39:32.625334 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.625311 2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:39:32.627219 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.627198 2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:39:32.644475 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.644452 2558 log.go:25] "Validated CRI v1 runtime API" Jun 02 21:39:32.651880 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.651859 2558 log.go:25] "Validated CRI v1 image API" Jun 02 21:39:32.653302 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.653283 2558 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 21:39:32.658500 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.658474 2558 fs.go:135] Filesystem UUIDs: map[72b810d3-7329-4f76-ae19-46496eac0435:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 efc54539-0a6f-4e9b-a577-46fd715dedff:/dev/nvme0n1p4] Jun 02 21:39:32.658581 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.658498 2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 21:39:32.662578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.661893 2558 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:39:32.664505 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.664384 2558 manager.go:217] Machine: {Timestamp:2026-06-02 21:39:32.663867738 +0000 UTC m=+0.465228703 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2499996 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ab12075ac4fef0bdd0a08567f66aa SystemUUID:ec2ab120-75ac-4fef-0bdd-0a08567f66aa BootID:161f14ad-e498-4f96-ba9c-d7d9f7c0a0cc Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:de:5b:cc:93:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:de:5b:cc:93:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:31:ce:df:f3:c8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 21:39:32.665283 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.665272 2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 21:39:32.665377 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.665364 2558 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 21:39:32.667477 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.667443 2558 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 21:39:32.667633 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.667481 2558 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 21:39:32.667677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.667643 2558 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 21:39:32.667677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.667653 2558 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 21:39:32.667677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.667662 2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:39:32.668327 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.668315 2558 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:39:32.670391 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.670376 2558 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:39:32.670509 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.670500 2558 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 21:39:32.673820 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.673804 2558 kubelet.go:491] "Attempting to sync node with API server" Jun 02 21:39:32.673820 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.673824 2558 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 21:39:32.673921 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.673841 2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 21:39:32.673921 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.673854 2558 kubelet.go:397] "Adding apiserver pod source" Jun 02 21:39:32.673921 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.673862 2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 21:39:32.675127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.675114 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:39:32.675174 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.675134 2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:39:32.678878 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.678857 2558 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 21:39:32.681378 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.681359 2558 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 21:39:32.684361 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684345 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 21:39:32.684361 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684363 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684370 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684375 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684385 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684391 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684400 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684409 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684419 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684427 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684436 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 21:39:32.684483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.684442 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 21:39:32.686423 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.686410 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 21:39:32.686423 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.686424 2558 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 21:39:32.687485 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.687459 2558 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 21:39:32.687556 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.687460 2558 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 21:39:32.689666 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.689652 2558 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 21:39:32.689736 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.689693 2558 server.go:1295] "Started kubelet" Jun 02 21:39:32.689795 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.689759 2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 21:39:32.689836 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.689789 2558 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 21:39:32.689891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.689859 2558 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 21:39:32.690655 ip-10-0-143-61 systemd[1]: Started Kubernetes Kubelet. Jun 02 21:39:32.692135 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.692118 2558 server.go:317] "Adding debug handlers to kubelet server" Jun 02 21:39:32.693809 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.693793 2558 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 21:39:32.698482 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.698456 2558 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 21:39:32.699216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.699198 2558 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-61.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 21:39:32.699733 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.699719 2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 21:39:32.699929 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.698839 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-61.ec2.internal.18b56172a35a3df3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-61.ec2.internal,UID:ip-10-0-143-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-61.ec2.internal,},FirstTimestamp:2026-06-02 21:39:32.689665523 +0000 UTC m=+0.491026489,LastTimestamp:2026-06-02 21:39:32.689665523 +0000 UTC m=+0.491026489,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-61.ec2.internal,}" Jun 02 21:39:32.700335 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700320 2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 21:39:32.700493 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700479 2558 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 21:39:32.700493 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700493 2558 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 21:39:32.700617 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700527 2558 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 21:39:32.700664 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700658 2558 reconstruct.go:97] "Volume reconstruction finished" Jun 02 21:39:32.700717 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.700668 2558 reconciler.go:26] "Reconciler: start to sync state" Jun 02 21:39:32.701266 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701250 2558 factory.go:55] Registering systemd factory Jun 02 21:39:32.701338 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701270 2558 factory.go:223] Registration of the systemd container factory successfully Jun 02 21:39:32.701397 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.701372 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:32.701500 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701488 2558 factory.go:153] Registering CRI-O factory Jun 02 21:39:32.701540 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701502 2558 factory.go:223] Registration of the crio container factory successfully Jun 02 21:39:32.701580 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701554 2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 21:39:32.701636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701580 2558 factory.go:103] Registering Raw factory Jun 02 21:39:32.701636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.701594 2558 manager.go:1196] Started watching for new ooms in manager Jun 02 21:39:32.702214 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.702195 2558 manager.go:319] Starting recovery of all containers Jun 02 21:39:32.706925 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.706891 2558 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 21:39:32.707076 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.707045 2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 21:39:32.714934 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.714903 2558 manager.go:324] Recovery completed Jun 02 21:39:32.719425 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.719402 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j5s5r" Jun 02 21:39:32.720214 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.720169 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.721899 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.721881 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.721980 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.721916 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.721980 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.721929 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.722519 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.722506 2558 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 21:39:32.722519 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.722518 2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 21:39:32.722588 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.722534 2558 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:39:32.724150 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.724078 2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-61.ec2.internal.18b56172a5460eed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-61.ec2.internal,UID:ip-10-0-143-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-61.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-61.ec2.internal,},FirstTimestamp:2026-06-02 21:39:32.721897197 +0000 UTC m=+0.523258168,LastTimestamp:2026-06-02 21:39:32.721897197 +0000 UTC m=+0.523258168,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-61.ec2.internal,}" Jun 02 21:39:32.725542 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.725526 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-j5s5r" Jun 02 21:39:32.726769 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.726757 2558 policy_none.go:49] "None policy: Start" Jun 02 21:39:32.726819 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.726774 2558 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 21:39:32.726819 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.726785 2558 state_mem.go:35] "Initializing new in-memory state store" Jun 02 21:39:32.773943 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.773774 2558 manager.go:341] "Starting Device Plugin manager" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.773964 2558 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.773975 2558 server.go:85] "Starting device plugin registration server" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.774219 2558 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.774230 2558 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.774330 2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.774421 2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.774430 2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.774959 2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 21:39:32.788622 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.775023 2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:32.804441 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.804402 2558 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 21:39:32.805663 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.805644 2558 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 21:39:32.805761 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.805670 2558 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 21:39:32.805761 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.805691 2558 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 21:39:32.805761 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.805697 2558 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 21:39:32.805761 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.805745 2558 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 21:39:32.808423 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.808402 2558 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:32.875447 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.875359 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.877276 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.877254 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.877391 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.877289 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.877391 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.877304 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.877391 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.877335 2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.887835 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.887812 2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.887835 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.887837 2558 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-61.ec2.internal\": node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:32.904955 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.904925 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:32.905939 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.905919 2558 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal"] Jun 02 21:39:32.906034 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.906019 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.907441 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.907426 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.907527 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.907453 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.907527 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.907463 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.908510 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.908497 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.908676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.908660 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.908747 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.908698 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.909220 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909206 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.909296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909218 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.909296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909242 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.909296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909252 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.909296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909226 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.909467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.909315 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.911013 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.910983 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.911099 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.911027 2558 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:39:32.911634 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.911621 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:39:32.911702 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.911643 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:39:32.911702 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:32.911653 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:39:32.926541 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.926520 2558 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-61.ec2.internal\" not found" node="ip-10-0-143-61.ec2.internal" Jun 02 21:39:32.931048 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:32.931032 2558 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-61.ec2.internal\" not found" node="ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.003129 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.003090 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.003129 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.003125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.003333 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.003144 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23c71c5268172c8479e10de3f191f37f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-61.ec2.internal\" (UID: \"23c71c5268172c8479e10de3f191f37f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.005197 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.005177 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.103640 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.103605 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.103735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.103641 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.103735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.103660 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23c71c5268172c8479e10de3f191f37f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-61.ec2.internal\" (UID: \"23c71c5268172c8479e10de3f191f37f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.103735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.103707 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.103859 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.103780 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/23c71c5268172c8479e10de3f191f37f-config\") pod \"kube-apiserver-proxy-ip-10-0-143-61.ec2.internal\" (UID: \"23c71c5268172c8479e10de3f191f37f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.105505 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.104008 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bb21253b0baf1f010740169823cb600b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal\" (UID: \"bb21253b0baf1f010740169823cb600b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.105505 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.105469 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.206400 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.206306 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.229503 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.229465 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.235072 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.235054 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.306983 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.306941 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.407481 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.407437 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.508029 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.507923 2558 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-143-61.ec2.internal\" not found" Jun 02 21:39:33.524828 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.524807 2558 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:33.596241 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.596200 2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 21:39:33.596677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.596355 2558 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:39:33.596677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.596406 2558 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:39:33.601387 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.601360 2558 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.622901 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.622877 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:39:33.624774 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.624759 2558 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" Jun 02 21:39:33.637399 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.637376 2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:39:33.674597 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.674566 2558 apiserver.go:52] "Watching apiserver" Jun 02 21:39:33.691030 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.690999 2558 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 21:39:33.691344 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.691322 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-tjhms","openshift-network-diagnostics/network-check-target-4q595","openshift-network-operator/iptables-alerter-jgn4q","openshift-ovn-kubernetes/ovnkube-node-rwwhv","kube-system/konnectivity-agent-srmjb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29","openshift-multus/multus-additional-cni-plugins-2k9mn","openshift-multus/network-metrics-daemon-s6gxd","kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal","openshift-cluster-node-tuning-operator/tuned-znd62","openshift-image-registry/node-ca-npgt5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal"] Jun 02 21:39:33.693584 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.693568 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:33.693660 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.693638 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:33.695455 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.695439 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.697471 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.697450 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.699162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.699144 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 21:39:33.699252 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.699144 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 21:39:33.699252 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.699207 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.699726 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.699709 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.699802 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.699788 2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 21:39:33.699877 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.699780 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:33.701685 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.701666 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.701789 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.701707 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 21:39:33.701789 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.701757 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 21:39:33.701899 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.701837 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 21:39:33.702101 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.702087 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lh942\"" Jun 02 21:39:33.702832 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.702819 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 21:39:33.703271 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703257 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.703310 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703302 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.703345 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703325 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.703345 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703341 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8g56b\"" Jun 02 21:39:33.703425 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703392 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 21:39:33.703723 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.703711 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.705629 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.705610 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 21:39:33.705846 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.705821 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.705911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.705828 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.706046 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706029 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.706444 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706424 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.706526 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-system-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.706526 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706470 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.706526 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706492 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-netd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706542 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdmrt\" (UniqueName: \"kubernetes.io/projected/afdad777-7975-4eef-bc20-ef9878174b53-kube-api-access-kdmrt\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706565 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-os-release\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706567 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pm2cm\"" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706579 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-bin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706597 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lq6f5\"" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-multus\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.706676 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706651 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706676 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-daemon-config\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706703 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706732 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-kubelet\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706756 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-systemd-units\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706781 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706806 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-system-cni-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706828 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-binary-copy\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706884 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-netns\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706909 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psb6r\" (UniqueName: \"kubernetes.io/projected/0f93e131-2668-45a1-ba9b-d207128c0de1-kube-api-access-psb6r\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706936 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-systemd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706958 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovn-node-metrics-cert\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.706982 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-script-lib\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707038 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707073 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-cnibin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707132 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-multus-certs\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707159 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-ovn\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707183 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-env-overrides\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707206 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707232 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hgx7\" (UniqueName: \"kubernetes.io/projected/744ab60f-c16e-4f34-8d7a-879cde111ece-kube-api-access-4hgx7\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707259 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-k8s-cni-cncf-io\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707304 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-etc-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707340 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707370 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-cni-binary-copy\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707394 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-kubelet\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707416 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-etc-kubernetes\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707450 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-netns\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-bin\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707490 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707494 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-cnibin\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707512 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-os-release\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707528 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-socket-dir-parent\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707541 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-config\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707574 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707594 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-hostroot\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707609 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-conf-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707623 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-slash\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707644 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-var-lib-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707658 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-node-log\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707671 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-log-socket\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707687 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.707955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.707726 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvnp\" (UniqueName: \"kubernetes.io/projected/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-kube-api-access-zpvnp\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.709246 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.708925 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.711625 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.711040 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jdh7g\"" Jun 02 21:39:33.711625 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.711288 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 21:39:33.711625 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.711305 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 21:39:33.712031 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.711894 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.712399 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.712373 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.712842 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.712827 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.713497 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.713473 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 21:39:33.713636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.713621 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sk2tz\"" Jun 02 21:39:33.715542 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.715523 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.718510 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.718493 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.718752 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.718737 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-b9p9q\"" Jun 02 21:39:33.718817 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.718808 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 21:39:33.719025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.719013 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 21:39:33.719395 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.719383 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.719595 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.719582 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:39:33.719595 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.719589 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b8nqm\"" Jun 02 21:39:33.724691 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.724674 2558 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:39:33.727268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.727233 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 21:34:32 +0000 UTC" deadline="2028-02-14 04:29:25.365492142 +0000 UTC" Jun 02 21:39:33.727268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.727264 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14910h49m51.638230045s" Jun 02 21:39:33.748540 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.748511 2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fzssh" Jun 02 21:39:33.757154 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.757132 2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fzssh" Jun 02 21:39:33.786269 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:33.786236 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb21253b0baf1f010740169823cb600b.slice/crio-b101b6e121b8da2b12f36e73b16f2f051612aac4ca5d03c089e1185c610e38f7 WatchSource:0}: Error finding container b101b6e121b8da2b12f36e73b16f2f051612aac4ca5d03c089e1185c610e38f7: Status 404 returned error can't find the container with id b101b6e121b8da2b12f36e73b16f2f051612aac4ca5d03c089e1185c610e38f7 Jun 02 21:39:33.786531 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:33.786516 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23c71c5268172c8479e10de3f191f37f.slice/crio-55bcabcf20dd92b19db800c16b40f5aa36827dd0d978dfdbbb2ec767cd47563d WatchSource:0}: Error finding container 55bcabcf20dd92b19db800c16b40f5aa36827dd0d978dfdbbb2ec767cd47563d: Status 404 returned error can't find the container with id 55bcabcf20dd92b19db800c16b40f5aa36827dd0d978dfdbbb2ec767cd47563d Jun 02 21:39:33.791037 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.791019 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:39:33.801472 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.801453 2558 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 21:39:33.807911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.807889 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysconfig\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.808015 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.807923 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-tuned\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.808015 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.807946 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-netns\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808015 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.807963 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-bin\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808015 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808000 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-cnibin\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808024 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-os-release\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808050 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-socket-dir-parent\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808076 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-host\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808133 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-cnibin\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808158 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-os-release\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.808204 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808192 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-config\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808209 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-socket-dir-parent\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808227 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808257 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7lf8\" (UniqueName: \"kubernetes.io/projected/c7859d70-6355-4231-89d0-5b0fa8e8a79c-kube-api-access-m7lf8\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808269 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-bin\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808290 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-node-log\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808296 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-netns\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808316 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-log-socket\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808341 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvnp\" (UniqueName: \"kubernetes.io/projected/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-kube-api-access-zpvnp\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808365 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-os-release\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808390 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808391 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-multus-certs\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808420 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808447 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808449 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-log-socket\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.808479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808473 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-netd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808499 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kdmrt\" (UniqueName: \"kubernetes.io/projected/afdad777-7975-4eef-bc20-ef9878174b53-kube-api-access-kdmrt\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808505 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-node-log\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808526 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-multus\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808551 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-psb6r\" (UniqueName: \"kubernetes.io/projected/0f93e131-2668-45a1-ba9b-d207128c0de1-kube-api-access-psb6r\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808577 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ac0155-f9f6-4447-84a7-8ebad399a628-host-slash\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808663 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9msqs\" (UniqueName: \"kubernetes.io/projected/b7ac0155-f9f6-4447-84a7-8ebad399a628-kube-api-access-9msqs\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808649 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" event={"ID":"bb21253b0baf1f010740169823cb600b","Type":"ContainerStarted","Data":"b101b6e121b8da2b12f36e73b16f2f051612aac4ca5d03c089e1185c610e38f7"} Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808692 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808725 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-kubelet\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808747 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-multus\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808760 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808774 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-cni-netd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808830 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-os-release\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808928 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808935 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-system-cni-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808962 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-kubelet\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809156 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808967 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808930 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-multus-certs\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.808967 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-netns\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809063 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-system-cni-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809093 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-config\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809093 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-kubernetes\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809142 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-netns\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809165 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-systemd\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809193 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-systemd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809208 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovn-node-metrics-cert\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809251 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-systemd\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809285 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809313 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-sys\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809337 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-lib-modules\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809362 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-k8s-cni-cncf-io\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.809432 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809475 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-sys-fs\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.809960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809521 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-run-k8s-cni-cncf-io\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.809536 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:34.309485546 +0000 UTC m=+2.110846508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809582 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-kubelet\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7ac0155-f9f6-4447-84a7-8ebad399a628-iptables-alerter-script\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809622 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-var-lib-kubelet\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809631 2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809647 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84c4p\" (UniqueName: \"kubernetes.io/projected/15943a77-86db-4d74-88c5-57b1c2e522ea-kube-api-access-84c4p\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809672 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-kubelet\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809675 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-hostroot\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809706 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-hostroot\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809711 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-conf-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809741 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-device-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809743 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-conf-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809775 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d35d06ad-ad18-49ea-8889-caafae090e7b-agent-certs\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809797 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-conf\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809822 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-slash\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809845 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-var-lib-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.811594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809862 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809881 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809930 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-var-lib-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.809930 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810019 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-host-slash\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810021 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-system-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810067 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810068 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-system-cni-dir\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810091 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-tmp\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810142 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-bin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810172 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-daemon-config\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810202 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810212 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-host-var-lib-cni-bin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810227 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d35d06ad-ad18-49ea-8889-caafae090e7b-konnectivity-ca\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810255 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-run\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810282 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rv5q\" (UniqueName: \"kubernetes.io/projected/e07f8f4f-913d-45b0-87cf-6880ff15d753-kube-api-access-7rv5q\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810305 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-socket-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.812100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810365 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-systemd-units\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810390 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-binary-copy\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810431 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7859d70-6355-4231-89d0-5b0fa8e8a79c-host\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810465 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-script-lib\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810472 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-systemd-units\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810456 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810492 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-cnibin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810536 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-modprobe-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810547 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-cnibin\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810567 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-ovn\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810595 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-env-overrides\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810636 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-run-ovn\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810649 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810680 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4hgx7\" (UniqueName: \"kubernetes.io/projected/744ab60f-c16e-4f34-8d7a-879cde111ece-kube-api-access-4hgx7\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810730 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7859d70-6355-4231-89d0-5b0fa8e8a79c-serviceca\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810735 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-multus-daemon-config\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810759 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-etc-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.812550 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810834 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810881 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-cni-binary-copy\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810908 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-etc-kubernetes\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810910 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-etc-openvswitch\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.810959 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-registration-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811054 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0f93e131-2668-45a1-ba9b-d207128c0de1-etc-kubernetes\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811192 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-binary-copy\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811221 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/744ab60f-c16e-4f34-8d7a-879cde111ece-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811220 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-env-overrides\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811257 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovnkube-script-lib\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811308 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/744ab60f-c16e-4f34-8d7a-879cde111ece-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811428 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0f93e131-2668-45a1-ba9b-d207128c0de1-cni-binary-copy\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.813038 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.811726 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" event={"ID":"23c71c5268172c8479e10de3f191f37f","Type":"ContainerStarted","Data":"55bcabcf20dd92b19db800c16b40f5aa36827dd0d978dfdbbb2ec767cd47563d"} Jun 02 21:39:33.813424 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.813281 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-ovn-node-metrics-cert\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.817530 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.817511 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:33.817621 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.817534 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:33.817621 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.817546 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:33.817723 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:33.817627 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:34.317594392 +0000 UTC m=+2.118955360 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:33.819869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.819851 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvnp\" (UniqueName: \"kubernetes.io/projected/bc2174cc-1c22-45c9-987f-5cd42bbfa9a5-kube-api-access-zpvnp\") pod \"ovnkube-node-rwwhv\" (UID: \"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:33.819950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.819903 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psb6r\" (UniqueName: \"kubernetes.io/projected/0f93e131-2668-45a1-ba9b-d207128c0de1-kube-api-access-psb6r\") pod \"multus-tjhms\" (UID: \"0f93e131-2668-45a1-ba9b-d207128c0de1\") " pod="openshift-multus/multus-tjhms" Jun 02 21:39:33.819950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.819909 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdmrt\" (UniqueName: \"kubernetes.io/projected/afdad777-7975-4eef-bc20-ef9878174b53-kube-api-access-kdmrt\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:33.824646 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.824626 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hgx7\" (UniqueName: \"kubernetes.io/projected/744ab60f-c16e-4f34-8d7a-879cde111ece-kube-api-access-4hgx7\") pod \"multus-additional-cni-plugins-2k9mn\" (UID: \"744ab60f-c16e-4f34-8d7a-879cde111ece\") " pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:33.845612 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.845572 2558 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:33.911870 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911826 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-84c4p\" (UniqueName: \"kubernetes.io/projected/15943a77-86db-4d74-88c5-57b1c2e522ea-kube-api-access-84c4p\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.911870 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911866 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-device-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911883 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d35d06ad-ad18-49ea-8889-caafae090e7b-agent-certs\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911898 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-conf\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911922 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911947 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-tmp\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911973 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.911974 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-device-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912016 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d35d06ad-ad18-49ea-8889-caafae090e7b-konnectivity-ca\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912081 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912105 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912125 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-run\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912152 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rv5q\" (UniqueName: \"kubernetes.io/projected/e07f8f4f-913d-45b0-87cf-6880ff15d753-kube-api-access-7rv5q\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912176 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-socket-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912205 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7859d70-6355-4231-89d0-5b0fa8e8a79c-host\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912219 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-run\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912231 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-modprobe-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912233 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysctl-conf\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912258 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7859d70-6355-4231-89d0-5b0fa8e8a79c-serviceca\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912274 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c7859d70-6355-4231-89d0-5b0fa8e8a79c-host\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912288 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-registration-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912309 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysconfig\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912330 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-tuned\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912353 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-socket-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912364 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-modprobe-d\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912356 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-host\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912393 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-host\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912422 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m7lf8\" (UniqueName: \"kubernetes.io/projected/c7859d70-6355-4231-89d0-5b0fa8e8a79c-kube-api-access-m7lf8\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912414 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-registration-dir\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.912576 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912456 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912503 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ac0155-f9f6-4447-84a7-8ebad399a628-host-slash\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912517 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-sysconfig\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912528 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9msqs\" (UniqueName: \"kubernetes.io/projected/b7ac0155-f9f6-4447-84a7-8ebad399a628-kube-api-access-9msqs\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912582 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-kubernetes\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912601 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912605 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d35d06ad-ad18-49ea-8889-caafae090e7b-konnectivity-ca\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912609 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-systemd\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912673 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-systemd\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912685 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-sys\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912731 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b7ac0155-f9f6-4447-84a7-8ebad399a628-host-slash\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912810 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-lib-modules\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912816 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-lib-modules\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912844 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-sys-fs\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912882 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7ac0155-f9f6-4447-84a7-8ebad399a628-iptables-alerter-script\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912889 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-kubernetes\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912893 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c7859d70-6355-4231-89d0-5b0fa8e8a79c-serviceca\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912845 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-sys\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.913409 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912904 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/15943a77-86db-4d74-88c5-57b1c2e522ea-sys-fs\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:33.914337 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.912908 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-var-lib-kubelet\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.914337 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.913043 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07f8f4f-913d-45b0-87cf-6880ff15d753-var-lib-kubelet\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.914337 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.913393 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b7ac0155-f9f6-4447-84a7-8ebad399a628-iptables-alerter-script\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.914475 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.914409 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-etc-tuned\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.914614 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.914598 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e07f8f4f-913d-45b0-87cf-6880ff15d753-tmp\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.914678 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.914658 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d35d06ad-ad18-49ea-8889-caafae090e7b-agent-certs\") pod \"konnectivity-agent-srmjb\" (UID: \"d35d06ad-ad18-49ea-8889-caafae090e7b\") " pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:33.921285 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.921252 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9msqs\" (UniqueName: \"kubernetes.io/projected/b7ac0155-f9f6-4447-84a7-8ebad399a628-kube-api-access-9msqs\") pod \"iptables-alerter-jgn4q\" (UID: \"b7ac0155-f9f6-4447-84a7-8ebad399a628\") " pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:33.921285 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.921277 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7lf8\" (UniqueName: \"kubernetes.io/projected/c7859d70-6355-4231-89d0-5b0fa8e8a79c-kube-api-access-m7lf8\") pod \"node-ca-npgt5\" (UID: \"c7859d70-6355-4231-89d0-5b0fa8e8a79c\") " pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:33.921436 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.921291 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rv5q\" (UniqueName: \"kubernetes.io/projected/e07f8f4f-913d-45b0-87cf-6880ff15d753-kube-api-access-7rv5q\") pod \"tuned-znd62\" (UID: \"e07f8f4f-913d-45b0-87cf-6880ff15d753\") " pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:33.921612 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:33.921594 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84c4p\" (UniqueName: \"kubernetes.io/projected/15943a77-86db-4d74-88c5-57b1c2e522ea-kube-api-access-84c4p\") pod \"aws-ebs-csi-driver-node-gsz29\" (UID: \"15943a77-86db-4d74-88c5-57b1c2e522ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:34.023107 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.023023 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:34.028910 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.028889 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" Jun 02 21:39:34.029246 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.029221 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2174cc_1c22_45c9_987f_5cd42bbfa9a5.slice/crio-e2c157ca142fd5006038d3baeba86b9b1ff82c91c9d92895ee58ef74ec5fd76b WatchSource:0}: Error finding container e2c157ca142fd5006038d3baeba86b9b1ff82c91c9d92895ee58ef74ec5fd76b: Status 404 returned error can't find the container with id e2c157ca142fd5006038d3baeba86b9b1ff82c91c9d92895ee58ef74ec5fd76b Jun 02 21:39:34.035306 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.035283 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tjhms" Jun 02 21:39:34.035973 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.035952 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744ab60f_c16e_4f34_8d7a_879cde111ece.slice/crio-6528ad0ef61420243ddf584897a2c254d22d15bd2388b850d3ae31abac0a181e WatchSource:0}: Error finding container 6528ad0ef61420243ddf584897a2c254d22d15bd2388b850d3ae31abac0a181e: Status 404 returned error can't find the container with id 6528ad0ef61420243ddf584897a2c254d22d15bd2388b850d3ae31abac0a181e Jun 02 21:39:34.041509 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.041487 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f93e131_2668_45a1_ba9b_d207128c0de1.slice/crio-f563f336f5917f94e18d36c29c678a039752f28598b623bcd6918928152a2492 WatchSource:0}: Error finding container f563f336f5917f94e18d36c29c678a039752f28598b623bcd6918928152a2492: Status 404 returned error can't find the container with id f563f336f5917f94e18d36c29c678a039752f28598b623bcd6918928152a2492 Jun 02 21:39:34.071175 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.071142 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jgn4q" Jun 02 21:39:34.078014 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.077966 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ac0155_f9f6_4447_84a7_8ebad399a628.slice/crio-93b090ee5dd422d641b653ea13d015fbc768946fcf02ffdb284c68860c08cbc1 WatchSource:0}: Error finding container 93b090ee5dd422d641b653ea13d015fbc768946fcf02ffdb284c68860c08cbc1: Status 404 returned error can't find the container with id 93b090ee5dd422d641b653ea13d015fbc768946fcf02ffdb284c68860c08cbc1 Jun 02 21:39:34.092895 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.092862 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:34.098615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.098589 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" Jun 02 21:39:34.099347 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.099327 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35d06ad_ad18_49ea_8889_caafae090e7b.slice/crio-aa03781bfd5ee4aa3cda30d09e1d0a05d41321ba966c5026195a3177d710d446 WatchSource:0}: Error finding container aa03781bfd5ee4aa3cda30d09e1d0a05d41321ba966c5026195a3177d710d446: Status 404 returned error can't find the container with id aa03781bfd5ee4aa3cda30d09e1d0a05d41321ba966c5026195a3177d710d446 Jun 02 21:39:34.104049 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.104024 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-znd62" Jun 02 21:39:34.104832 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.104812 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15943a77_86db_4d74_88c5_57b1c2e522ea.slice/crio-5c7e72bed4d2624ce6df49db899637dd5584bb6936144e306a8dc7d58af5ea45 WatchSource:0}: Error finding container 5c7e72bed4d2624ce6df49db899637dd5584bb6936144e306a8dc7d58af5ea45: Status 404 returned error can't find the container with id 5c7e72bed4d2624ce6df49db899637dd5584bb6936144e306a8dc7d58af5ea45 Jun 02 21:39:34.109546 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.109527 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-npgt5" Jun 02 21:39:34.110018 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.109976 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07f8f4f_913d_45b0_87cf_6880ff15d753.slice/crio-77bf9bbb9932882fb9fda09548a9754790e87f9cf3b92c5f97597e5147394ba0 WatchSource:0}: Error finding container 77bf9bbb9932882fb9fda09548a9754790e87f9cf3b92c5f97597e5147394ba0: Status 404 returned error can't find the container with id 77bf9bbb9932882fb9fda09548a9754790e87f9cf3b92c5f97597e5147394ba0 Jun 02 21:39:34.115680 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:39:34.115639 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7859d70_6355_4231_89d0_5b0fa8e8a79c.slice/crio-09b1161dda34d67ebb6bc0f0a7fadfeeaa528af5efde6863d1c6d7b3ea2b5f0e WatchSource:0}: Error finding container 09b1161dda34d67ebb6bc0f0a7fadfeeaa528af5efde6863d1c6d7b3ea2b5f0e: Status 404 returned error can't find the container with id 09b1161dda34d67ebb6bc0f0a7fadfeeaa528af5efde6863d1c6d7b3ea2b5f0e Jun 02 21:39:34.242624 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.242595 2558 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:34.316532 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.316440 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:34.316670 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.316592 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:34.316729 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.316671 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:35.316650017 +0000 UTC m=+3.118010983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:34.416962 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.416869 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:34.417141 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.417087 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:34.417141 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.417118 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:34.417141 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.417133 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:34.417264 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.417195 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:35.417175961 +0000 UTC m=+3.218536930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:34.758456 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.758363 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:34:33 +0000 UTC" deadline="2028-01-19 07:01:28.280528631 +0000 UTC" Jun 02 21:39:34.758456 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.758404 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14289h21m53.522127627s" Jun 02 21:39:34.806824 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.806791 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:34.807013 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:34.806939 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:34.819528 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.819490 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-npgt5" event={"ID":"c7859d70-6355-4231-89d0-5b0fa8e8a79c","Type":"ContainerStarted","Data":"09b1161dda34d67ebb6bc0f0a7fadfeeaa528af5efde6863d1c6d7b3ea2b5f0e"} Jun 02 21:39:34.828342 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.828299 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" event={"ID":"15943a77-86db-4d74-88c5-57b1c2e522ea","Type":"ContainerStarted","Data":"5c7e72bed4d2624ce6df49db899637dd5584bb6936144e306a8dc7d58af5ea45"} Jun 02 21:39:34.841948 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.841913 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-srmjb" event={"ID":"d35d06ad-ad18-49ea-8889-caafae090e7b","Type":"ContainerStarted","Data":"aa03781bfd5ee4aa3cda30d09e1d0a05d41321ba966c5026195a3177d710d446"} Jun 02 21:39:34.856292 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.856211 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tjhms" event={"ID":"0f93e131-2668-45a1-ba9b-d207128c0de1","Type":"ContainerStarted","Data":"f563f336f5917f94e18d36c29c678a039752f28598b623bcd6918928152a2492"} Jun 02 21:39:34.868791 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.868755 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"e2c157ca142fd5006038d3baeba86b9b1ff82c91c9d92895ee58ef74ec5fd76b"} Jun 02 21:39:34.870464 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.870434 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-znd62" event={"ID":"e07f8f4f-913d-45b0-87cf-6880ff15d753","Type":"ContainerStarted","Data":"77bf9bbb9932882fb9fda09548a9754790e87f9cf3b92c5f97597e5147394ba0"} Jun 02 21:39:34.873755 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.873727 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jgn4q" event={"ID":"b7ac0155-f9f6-4447-84a7-8ebad399a628","Type":"ContainerStarted","Data":"93b090ee5dd422d641b653ea13d015fbc768946fcf02ffdb284c68860c08cbc1"} Jun 02 21:39:34.884087 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:34.883946 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerStarted","Data":"6528ad0ef61420243ddf584897a2c254d22d15bd2388b850d3ae31abac0a181e"} Jun 02 21:39:35.178681 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.178562 2558 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:35.323364 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.323328 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:35.323572 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.323560 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:35.323634 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.323627 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:37.323607912 +0000 UTC m=+5.124968875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:35.425335 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.424656 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:35.425335 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.424842 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:35.425335 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.424863 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:35.425335 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.424883 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:35.425335 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.424945 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:37.424926208 +0000 UTC m=+5.226287166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:35.759221 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.759172 2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:34:33 +0000 UTC" deadline="2028-01-20 14:00:57.841109712 +0000 UTC" Jun 02 21:39:35.759221 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.759218 2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14320h21m22.081895588s" Jun 02 21:39:35.805922 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:35.805890 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:35.806128 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:35.806036 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:36.586208 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:36.586176 2558 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:39:36.810711 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:36.810029 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:36.810711 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:36.810353 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:37.341738 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:37.341182 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:37.341738 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.341333 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:37.341738 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.341396 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:41.341377686 +0000 UTC m=+9.142738647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:37.441611 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:37.441573 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:37.441801 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.441772 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:37.441801 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.441792 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:37.441914 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.441805 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:37.441914 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.441864 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:41.441844903 +0000 UTC m=+9.243205866 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:37.806717 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:37.806635 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:37.806897 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:37.806765 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:38.806210 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:38.806174 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:38.806773 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:38.806324 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:39.806354 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:39.806319 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:39.806856 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:39.806449 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:40.324126 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.324092 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-5mqkj"] Jun 02 21:39:40.330573 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.330538 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.333284 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.333254 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6wrhv\"" Jun 02 21:39:40.333469 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.333444 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 21:39:40.334208 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.334183 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 21:39:40.368553 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.368514 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/037ea9d6-87f1-492a-9026-58e056b65527-hosts-file\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.368734 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.368570 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scbsk\" (UniqueName: \"kubernetes.io/projected/037ea9d6-87f1-492a-9026-58e056b65527-kube-api-access-scbsk\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.368734 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.368679 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037ea9d6-87f1-492a-9026-58e056b65527-tmp-dir\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.470140 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.470096 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037ea9d6-87f1-492a-9026-58e056b65527-tmp-dir\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.470321 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.470193 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/037ea9d6-87f1-492a-9026-58e056b65527-hosts-file\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.470321 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.470220 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-scbsk\" (UniqueName: \"kubernetes.io/projected/037ea9d6-87f1-492a-9026-58e056b65527-kube-api-access-scbsk\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.470568 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.470545 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/037ea9d6-87f1-492a-9026-58e056b65527-hosts-file\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.471042 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.471014 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/037ea9d6-87f1-492a-9026-58e056b65527-tmp-dir\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.482622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.482590 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scbsk\" (UniqueName: \"kubernetes.io/projected/037ea9d6-87f1-492a-9026-58e056b65527-kube-api-access-scbsk\") pod \"node-resolver-5mqkj\" (UID: \"037ea9d6-87f1-492a-9026-58e056b65527\") " pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.642779 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.642521 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mqkj" Jun 02 21:39:40.806240 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:40.806204 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:40.806408 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:40.806322 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:41.376142 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:41.376108 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:41.376323 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.376243 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:41.376323 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.376314 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:49.376299032 +0000 UTC m=+17.177659987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:41.476820 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:41.476778 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:41.477042 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.477000 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:41.477042 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.477022 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:41.477042 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.477035 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:41.477205 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.477107 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:49.477086013 +0000 UTC m=+17.278446972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:41.806537 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:41.806442 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:41.806980 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:41.806571 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:42.810052 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:42.809951 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:42.810543 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:42.810156 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:43.806674 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:43.806638 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:43.806863 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:43.806770 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:44.808980 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.808955 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:44.809329 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:44.809079 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:44.835704 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.835677 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-tpm87"] Jun 02 21:39:44.867536 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.867497 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:44.867706 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:44.867584 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:44.904970 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.904929 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-kubelet-config\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:44.904970 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.904975 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-dbus\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:44.905194 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:44.905025 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006422 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.006331 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-dbus\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006422 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.006379 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006618 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.006443 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-kubelet-config\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006618 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.006527 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-kubelet-config\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006618 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.006560 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e6fcc949-f931-481e-8bce-c896aaaa5189-dbus\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.006618 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:45.006574 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:45.006776 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:45.006660 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:45.506637545 +0000 UTC m=+13.307998521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:45.511367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.511329 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:45.511553 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:45.511503 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:45.511626 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:45.511583 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:46.511560261 +0000 UTC m=+14.312921228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:45.806080 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:45.806004 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:45.806226 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:45.806126 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:46.518005 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:46.517947 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:46.518414 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:46.518127 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:46.518414 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:46.518201 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:48.518179556 +0000 UTC m=+16.319540523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:46.808694 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:46.808620 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:46.808854 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:46.808617 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:46.808854 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:46.808740 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:46.808854 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:46.808814 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:47.806802 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:47.806766 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:47.807292 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:47.806886 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:48.531534 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:48.531480 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:48.531705 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:48.531632 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:48.531761 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:48.531712 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:39:52.531691874 +0000 UTC m=+20.333052847 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:48.807003 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:48.806893 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:48.807003 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:48.806933 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:48.807514 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:48.807055 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:48.807514 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:48.807196 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:49.437854 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:49.437807 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:49.438111 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.437919 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:49.438111 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.438001 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:05.437966772 +0000 UTC m=+33.239327726 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:39:49.538737 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:49.538699 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:49.538928 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.538895 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:39:49.538928 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.538918 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:39:49.539061 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.538931 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:49.539061 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.539017 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:05.538979066 +0000 UTC m=+33.340340024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:39:49.806865 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:49.806776 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:49.807062 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:49.806912 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:50.806678 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:50.806642 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:50.806678 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:50.806678 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:50.810066 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:50.807138 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:50.810066 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:50.808907 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:51.806514 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:51.806475 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:51.806679 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:51.806611 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:52.560720 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:52.560686 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:52.561111 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:52.560835 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:52.561111 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:52.560913 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:00.560895965 +0000 UTC m=+28.362256924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:39:52.807007 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:52.806964 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:52.807167 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:52.807028 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:52.807167 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:52.807078 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:52.807167 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:52.807103 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:53.806698 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.806403 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:53.807260 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:53.806811 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:53.926769 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.926738 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="a6afda42a4b6882c841ab4192e0f6607a73c38769c10de5e570c5be5dd035a25" exitCode=0 Jun 02 21:39:53.926926 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.926811 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"a6afda42a4b6882c841ab4192e0f6607a73c38769c10de5e570c5be5dd035a25"} Jun 02 21:39:53.928249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.928192 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mqkj" event={"ID":"037ea9d6-87f1-492a-9026-58e056b65527","Type":"ContainerStarted","Data":"2d10ef310a28d06474424788c376a24614be0f0638cfc9e964ebb25bbee24b52"} Jun 02 21:39:53.928249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.928226 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mqkj" event={"ID":"037ea9d6-87f1-492a-9026-58e056b65527","Type":"ContainerStarted","Data":"8d3fa05f5da0e23beeb30ce65f24d04bda85bcabe3f583f1ae8d09fcc7cdc3d6"} Jun 02 21:39:53.929406 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.929384 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-npgt5" event={"ID":"c7859d70-6355-4231-89d0-5b0fa8e8a79c","Type":"ContainerStarted","Data":"c7abf95399454657ef322cff1ce8a170ffca74de2443394125e1407d1fef28a1"} Jun 02 21:39:53.930604 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.930581 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" event={"ID":"15943a77-86db-4d74-88c5-57b1c2e522ea","Type":"ContainerStarted","Data":"a1504c4e3162a81e91fbce332f79359b855b61d53dacfa6d39c48b0f7eb61645"} Jun 02 21:39:53.931786 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.931764 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-srmjb" event={"ID":"d35d06ad-ad18-49ea-8889-caafae090e7b","Type":"ContainerStarted","Data":"6c938efd5c0d8fb02e949221aadcfd55ab225b93787318c6259a628fdea70192"} Jun 02 21:39:53.933064 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.933043 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tjhms" event={"ID":"0f93e131-2668-45a1-ba9b-d207128c0de1","Type":"ContainerStarted","Data":"696f613b0ddad65b1afb9e13dadff7e37c538260baffaff19a767072eb48ea35"} Jun 02 21:39:53.935105 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.935080 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"3110991b5d6b921056a6e2643bdbdd26f88212d2f2f71fac4a6a8173b882c069"} Jun 02 21:39:53.935105 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.935107 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"8d8376de7e7f5344a41b9aeb0670351b6e0b72bf874425f1726d3be09da6a547"} Jun 02 21:39:53.935258 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.935117 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"a710aebafd5ebd24119bb10cece70f3bbdc9e1255fb3dfadc7994a5a6d763df3"} Jun 02 21:39:53.935258 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.935125 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"7cd5c679d31d44abf0d3201f69fc6693863859f3f78cadabbf5d23ee666c6f8e"} Jun 02 21:39:53.936311 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.936290 2558 generic.go:358] "Generic (PLEG): container finished" podID="bb21253b0baf1f010740169823cb600b" containerID="4c82c4f05f5f92036dfcfc2c6b2206ae94bac8e6059e35918e1321ac22f65887" exitCode=0 Jun 02 21:39:53.936384 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.936360 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" event={"ID":"bb21253b0baf1f010740169823cb600b","Type":"ContainerDied","Data":"4c82c4f05f5f92036dfcfc2c6b2206ae94bac8e6059e35918e1321ac22f65887"} Jun 02 21:39:53.937756 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.937738 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" event={"ID":"23c71c5268172c8479e10de3f191f37f","Type":"ContainerStarted","Data":"f5864cb5fbe9db2519df8ecc6b4b3cd8488c040860f561bc90f78d1a982f7ca5"} Jun 02 21:39:53.938909 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.938890 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-znd62" event={"ID":"e07f8f4f-913d-45b0-87cf-6880ff15d753","Type":"ContainerStarted","Data":"2ce721439a7d83d233c50beaa7950323709e713de7e4effbae7ace59f105908b"} Jun 02 21:39:53.984058 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.983975 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-npgt5" podStartSLOduration=1.931368961 podStartE2EDuration="20.983957683s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.116974812 +0000 UTC m=+1.918335768" lastFinishedPulling="2026-06-02 21:39:53.169563536 +0000 UTC m=+20.970924490" observedRunningTime="2026-06-02 21:39:53.972283506 +0000 UTC m=+21.773644482" watchObservedRunningTime="2026-06-02 21:39:53.983957683 +0000 UTC m=+21.785318657" Jun 02 21:39:53.984635 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:53.984610 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-61.ec2.internal" podStartSLOduration=20.984601665 podStartE2EDuration="20.984601665s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:39:53.984136433 +0000 UTC m=+21.785497391" watchObservedRunningTime="2026-06-02 21:39:53.984601665 +0000 UTC m=+21.785962640" Jun 02 21:39:54.002126 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.002076 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-srmjb" podStartSLOduration=10.269218201 podStartE2EDuration="21.002061053s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.101565973 +0000 UTC m=+1.902926926" lastFinishedPulling="2026-06-02 21:39:44.834408818 +0000 UTC m=+12.635769778" observedRunningTime="2026-06-02 21:39:54.001547488 +0000 UTC m=+21.802908463" watchObservedRunningTime="2026-06-02 21:39:54.002061053 +0000 UTC m=+21.803422027" Jun 02 21:39:54.036817 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.036778 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-znd62" podStartSLOduration=1.977359458 podStartE2EDuration="21.036765746s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.112365391 +0000 UTC m=+1.913726343" lastFinishedPulling="2026-06-02 21:39:53.171771666 +0000 UTC m=+20.973132631" observedRunningTime="2026-06-02 21:39:54.036214263 +0000 UTC m=+21.837575255" watchObservedRunningTime="2026-06-02 21:39:54.036765746 +0000 UTC m=+21.838126728" Jun 02 21:39:54.054446 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.054395 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-5mqkj" podStartSLOduration=14.054379814 podStartE2EDuration="14.054379814s" podCreationTimestamp="2026-06-02 21:39:40 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:39:54.05405934 +0000 UTC m=+21.855420314" watchObservedRunningTime="2026-06-02 21:39:54.054379814 +0000 UTC m=+21.855740789" Jun 02 21:39:54.807077 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.806857 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:54.807774 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.806854 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:54.807774 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:54.807191 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:54.807774 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:54.807258 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:54.944079 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.944032 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" event={"ID":"15943a77-86db-4d74-88c5-57b1c2e522ea","Type":"ContainerStarted","Data":"7f24244920d7d97658590ac85c1f91655cb12eb690a6f482f23a7b64441a3a68"} Jun 02 21:39:54.946543 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.946512 2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 21:39:54.948593 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.948502 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"84c37ba45739cdb45ece939c11055a0b8aa50b5c1e70ff1c40967da23487a3e7"} Jun 02 21:39:54.948593 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.948537 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"fcbdcaeae6d3e70ec1aa981c92be187ba1fa3b445d819b92a6b54821592b54da"} Jun 02 21:39:54.949882 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.949858 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jgn4q" event={"ID":"b7ac0155-f9f6-4447-84a7-8ebad399a628","Type":"ContainerStarted","Data":"fe062a64da40c444cb9ea1badde9e807ffaa7ea44fa0685a27ef3df3cf5edc25"} Jun 02 21:39:54.992415 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:54.992354 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-tjhms" podStartSLOduration=2.85928649 podStartE2EDuration="21.992333128s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.043243682 +0000 UTC m=+1.844604635" lastFinishedPulling="2026-06-02 21:39:53.176290309 +0000 UTC m=+20.977651273" observedRunningTime="2026-06-02 21:39:54.075465235 +0000 UTC m=+21.876826209" watchObservedRunningTime="2026-06-02 21:39:54.992333128 +0000 UTC m=+22.793694101" Jun 02 21:39:55.013483 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.013436 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jgn4q" podStartSLOduration=2.924061038 podStartE2EDuration="22.013421052s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.079597722 +0000 UTC m=+1.880958676" lastFinishedPulling="2026-06-02 21:39:53.16895773 +0000 UTC m=+20.970318690" observedRunningTime="2026-06-02 21:39:55.013312832 +0000 UTC m=+22.814674032" watchObservedRunningTime="2026-06-02 21:39:55.013421052 +0000 UTC m=+22.814782028" Jun 02 21:39:55.786128 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.785984 2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T21:39:54.946541036Z","UUID":"8b5750d4-7504-4957-be48-6d482238614c","Handler":null,"Name":"","Endpoint":""} Jun 02 21:39:55.789549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.789515 2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 21:39:55.789549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.789554 2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 21:39:55.806260 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.806226 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:55.806442 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:55.806361 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:55.953155 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.953114 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" event={"ID":"bb21253b0baf1f010740169823cb600b","Type":"ContainerStarted","Data":"d7ab938ad1d5ca1012d5360be74e74acd40b7c3559bbccb9b9da2f3ae2294f84"} Jun 02 21:39:55.971046 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:55.970974 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-61.ec2.internal" podStartSLOduration=22.970956671 podStartE2EDuration="22.970956671s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:39:55.970611237 +0000 UTC m=+23.771972213" watchObservedRunningTime="2026-06-02 21:39:55.970956671 +0000 UTC m=+23.772317647" Jun 02 21:39:56.806494 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:56.806453 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:56.806672 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:56.806602 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:56.806730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:56.806667 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:56.806814 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:56.806790 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:56.956896 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:56.956842 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" event={"ID":"15943a77-86db-4d74-88c5-57b1c2e522ea","Type":"ContainerStarted","Data":"85b90a86fecf5ecaec9f76e63fc1798f3fbeb4cf3c628d9182e1888c9aa00882"} Jun 02 21:39:56.959600 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:56.959575 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"4f076fe48aeabf909a578f27256f9c469aab47cc2e9c733547e88c61a5c92a17"} Jun 02 21:39:57.806179 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:57.806141 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:57.806366 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:57.806265 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:58.348759 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.348721 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:58.349474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.349439 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:58.374398 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.374343 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gsz29" podStartSLOduration=3.497398253 podStartE2EDuration="25.37432591s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.107112656 +0000 UTC m=+1.908473609" lastFinishedPulling="2026-06-02 21:39:55.984040297 +0000 UTC m=+23.785401266" observedRunningTime="2026-06-02 21:39:56.97926164 +0000 UTC m=+24.780622614" watchObservedRunningTime="2026-06-02 21:39:58.37432591 +0000 UTC m=+26.175686885" Jun 02 21:39:58.805947 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.805909 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:39:58.806144 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.805959 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:39:58.806144 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:58.806079 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:39:58.806263 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:58.806185 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:39:58.963824 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.963740 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:58.964363 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:58.964340 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-srmjb" Jun 02 21:39:59.806296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.806216 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:39:59.806872 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:39:59.806328 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:39:59.968220 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.968184 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" event={"ID":"bc2174cc-1c22-45c9-987f-5cd42bbfa9a5","Type":"ContainerStarted","Data":"b332bf34ddab433a98b56877236a3205635ec2eac32a9d454d7685bf940e2007"} Jun 02 21:39:59.968484 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.968465 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:59.968561 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.968495 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:39:59.969837 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.969810 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="d2ad30b562ce6245512912c50eacbea83bef34861b65bb28414df6fd4186da17" exitCode=0 Jun 02 21:39:59.969963 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.969892 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"d2ad30b562ce6245512912c50eacbea83bef34861b65bb28414df6fd4186da17"} Jun 02 21:39:59.984309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:39:59.984284 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:40:00.050082 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.050030 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" podStartSLOduration=8.562876175 podStartE2EDuration="28.05001109s" podCreationTimestamp="2026-06-02 21:39:32 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.030895339 +0000 UTC m=+1.832256295" lastFinishedPulling="2026-06-02 21:39:53.518030255 +0000 UTC m=+21.319391210" observedRunningTime="2026-06-02 21:40:00.010125726 +0000 UTC m=+27.811486701" watchObservedRunningTime="2026-06-02 21:40:00.05001109 +0000 UTC m=+27.851372090" Jun 02 21:40:00.629228 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.628594 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:00.629228 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:00.628757 2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:40:00.629228 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:00.628824 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret podName:e6fcc949-f931-481e-8bce-c896aaaa5189 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:16.628804337 +0000 UTC m=+44.430165298 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret") pod "global-pull-secret-syncer-tpm87" (UID: "e6fcc949-f931-481e-8bce-c896aaaa5189") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:40:00.806318 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.806284 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:00.806908 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.806325 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:00.806908 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:00.806409 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:40:00.806908 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:00.806584 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:40:00.974035 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.973813 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="45cab9ca65cfea024fa54c77a66475aa504bdb81935a9c98141ae550db2a58bd" exitCode=0 Jun 02 21:40:00.974035 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.973890 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"45cab9ca65cfea024fa54c77a66475aa504bdb81935a9c98141ae550db2a58bd"} Jun 02 21:40:00.974843 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.974799 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:40:00.990143 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:00.990098 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:40:01.264105 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.264022 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s6gxd"] Jun 02 21:40:01.264281 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.264125 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:01.264281 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:01.264214 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:40:01.267591 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.267560 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tpm87"] Jun 02 21:40:01.267749 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.267665 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:01.267817 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:01.267772 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:40:01.268187 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.268159 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4q595"] Jun 02 21:40:01.268285 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.268270 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:01.268420 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:01.268398 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:40:01.977807 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.977767 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="3f3ae0a61bfe6429efdecf02b7e676c7ef4cd4d755f27edf7595cf3d6fbe5c1c" exitCode=0 Jun 02 21:40:01.978267 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:01.977847 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"3f3ae0a61bfe6429efdecf02b7e676c7ef4cd4d755f27edf7595cf3d6fbe5c1c"} Jun 02 21:40:02.807325 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:02.807249 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:02.807463 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:02.807349 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:02.807463 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:02.807382 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:02.807463 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:02.807404 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:40:02.807463 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:02.807449 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:40:02.807591 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:02.807521 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:40:04.806788 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:04.806751 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:04.807209 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:04.806751 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:04.807209 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:04.806893 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4q595" podUID="279d6e77-7104-49f6-b3a4-df29f2bb64e2" Jun 02 21:40:04.807209 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:04.806975 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s6gxd" podUID="afdad777-7975-4eef-bc20-ef9878174b53" Jun 02 21:40:04.807209 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:04.806751 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:04.807209 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:04.807093 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tpm87" podUID="e6fcc949-f931-481e-8bce-c896aaaa5189" Jun 02 21:40:05.471009 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:05.470958 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:05.471218 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.471137 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:40:05.471285 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.471218 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:37.471197923 +0000 UTC m=+65.272558889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:40:05.572055 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:05.572023 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:05.572247 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.572218 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:40:05.572247 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.572246 2558 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:40:05.572328 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.572260 2558 projected.go:194] Error preparing data for projected volume kube-api-access-xlhdx for pod openshift-network-diagnostics/network-check-target-4q595: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:40:05.572368 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:05.572327 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx podName:279d6e77-7104-49f6-b3a4-df29f2bb64e2 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:37.572306633 +0000 UTC m=+65.373667591 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xlhdx" (UniqueName: "kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx") pod "network-check-target-4q595" (UID: "279d6e77-7104-49f6-b3a4-df29f2bb64e2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:40:06.033103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.033068 2558 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-143-61.ec2.internal" event="NodeReady" Jun 02 21:40:06.033485 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.033233 2558 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 21:40:06.098850 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.098808 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:40:06.121002 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.120951 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs"] Jun 02 21:40:06.121188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.121149 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.123732 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.123707 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 21:40:06.124845 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.124818 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 21:40:06.128172 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.128149 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 21:40:06.128314 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.128149 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrw4w\"" Jun 02 21:40:06.130070 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.130048 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 21:40:06.138763 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.138731 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm"] Jun 02 21:40:06.138938 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.138891 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.149184 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.149160 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 02 21:40:06.149612 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.149160 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.149693 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.149216 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 02 21:40:06.149803 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.149266 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.149874 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.149324 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-492qs\"" Jun 02 21:40:06.156929 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.156849 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv"] Jun 02 21:40:06.157091 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.157035 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" Jun 02 21:40:06.174590 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.174560 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.174789 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.174596 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.175655 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.175628 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7c6z9\"" Jun 02 21:40:06.183430 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.183340 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc"] Jun 02 21:40:06.183563 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.183530 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.186135 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.186103 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.186890 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.186859 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.187015 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.186898 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 02 21:40:06.190066 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.190027 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7c72w\"" Jun 02 21:40:06.203499 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.203467 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-88rls"] Jun 02 21:40:06.203651 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.203626 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.211058 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.211034 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jun 02 21:40:06.211257 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.211241 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-vb488\"" Jun 02 21:40:06.211350 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.211328 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jun 02 21:40:06.212735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.212568 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.217816 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.217792 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222010 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs"] Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222043 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222057 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm"] Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222069 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc"] Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222082 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8"] Jun 02 21:40:06.222188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.222122 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.229677 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.229650 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 02 21:40:06.229867 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.229774 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 02 21:40:06.229967 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.229948 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-2td4n\"" Jun 02 21:40:06.231215 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.230166 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.231215 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.230621 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.236611 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.236586 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 02 21:40:06.247213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.247172 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9"] Jun 02 21:40:06.247370 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.247299 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.251495 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.251471 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-25ljn\"" Jun 02 21:40:06.252322 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.252024 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 02 21:40:06.252322 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.252054 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 02 21:40:06.253227 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.253129 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.253348 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.253327 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.263359 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.263329 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k"] Jun 02 21:40:06.263580 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.263531 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.266751 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.266730 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 02 21:40:06.267397 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.267376 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 02 21:40:06.267513 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.267376 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 02 21:40:06.267513 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.267379 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 02 21:40:06.276182 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276152 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276196 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276244 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276296 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276275 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276368 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4ljz\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276413 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4wlt\" (UniqueName: \"kubernetes.io/projected/8796e9bb-846a-48ef-88be-45ac60ffc2fe-kube-api-access-t4wlt\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.276572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276446 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.276572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276499 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn6t7\" (UniqueName: \"kubernetes.io/projected/ec6d065e-18dc-4ce8-9741-5dc7242d0078-kube-api-access-gn6t7\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.276572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8796e9bb-846a-48ef-88be-45ac60ffc2fe-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.276572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276550 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8796e9bb-846a-48ef-88be-45ac60ffc2fe-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.276735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276577 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.276735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276600 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqr6w\" (UniqueName: \"kubernetes.io/projected/7f0e74e3-e3bf-4668-872f-d3b519382477-kube-api-access-cqr6w\") pod \"volume-data-source-validator-7598fd5bd-g5rnm\" (UID: \"7f0e74e3-e3bf-4668-872f-d3b519382477\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" Jun 02 21:40:06.276735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.276621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.285650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.285614 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv"] Jun 02 21:40:06.285650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.285658 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6"] Jun 02 21:40:06.285853 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.285769 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" Jun 02 21:40:06.292619 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.292593 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.293523 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.293492 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.293523 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.293505 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-d7jdk\"" Jun 02 21:40:06.305345 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305321 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9"] Jun 02 21:40:06.305345 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305347 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8"] Jun 02 21:40:06.305529 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305357 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-88rls"] Jun 02 21:40:06.305529 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305370 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k"] Jun 02 21:40:06.305529 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305381 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qhxs5"] Jun 02 21:40:06.305529 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.305482 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.310594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.310574 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 02 21:40:06.329043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.329005 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4pbgs"] Jun 02 21:40:06.329207 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.329121 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.337794 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.337772 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 21:40:06.337926 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.337823 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2vddp\"" Jun 02 21:40:06.337926 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.337862 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 21:40:06.344975 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.344950 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6"] Jun 02 21:40:06.345112 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.345041 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4pbgs"] Jun 02 21:40:06.345112 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.345057 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhxs5"] Jun 02 21:40:06.345199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.345128 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.349101 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.349071 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vfhfl\"" Jun 02 21:40:06.349221 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.349173 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 21:40:06.349471 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.349447 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 21:40:06.349594 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.349448 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 21:40:06.377071 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377034 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrzk2\" (UniqueName: \"kubernetes.io/projected/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-kube-api-access-rrzk2\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.377071 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377082 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-trusted-ca\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm6w6\" (UniqueName: \"kubernetes.io/projected/628721dd-9b4c-4673-9540-7457f195b066-kube-api-access-qm6w6\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377139 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gn6t7\" (UniqueName: \"kubernetes.io/projected/ec6d065e-18dc-4ce8-9741-5dc7242d0078-kube-api-access-gn6t7\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377164 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8796e9bb-846a-48ef-88be-45ac60ffc2fe-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377198 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377226 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/adf7837a-7e9f-4612-a770-52b31b7a06d9-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377255 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377306 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wlt\" (UniqueName: \"kubernetes.io/projected/8796e9bb-846a-48ef-88be-45ac60ffc2fe-kube-api-access-t4wlt\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377334 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.377367 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377360 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/658df708-29d1-44e2-83d5-33e5316847f7-serving-cert\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377396 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377425 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8796e9bb-846a-48ef-88be-45ac60ffc2fe-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377455 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377484 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckrsr\" (UniqueName: \"kubernetes.io/projected/5c59e95b-931d-4ff3-96a5-f1f06f21a168-kube-api-access-ckrsr\") pod \"network-check-source-56d84b9f44-k7m6k\" (UID: \"5c59e95b-931d-4ff3-96a5-f1f06f21a168\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377557 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/628721dd-9b4c-4673-9540-7457f195b066-serving-cert\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.377629 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.377645 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:06.377735 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.377709 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:06.877689706 +0000 UTC m=+34.679050664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377827 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377859 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377887 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/628721dd-9b4c-4673-9540-7457f195b066-config\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377924 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377955 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.377981 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-config\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.378083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.378031 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkgc8\" (UniqueName: \"kubernetes.io/projected/658df708-29d1-44e2-83d5-33e5316847f7-kube-api-access-fkgc8\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.378549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.378519 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8796e9bb-846a-48ef-88be-45ac60ffc2fe-config\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.378901 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.378575 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:06.378901 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.378610 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.378901 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.378782 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.378901 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.378857 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:06.878836677 +0000 UTC m=+34.680197643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379016 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4ljz\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379070 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379104 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379108 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrpm7\" (UniqueName: \"kubernetes.io/projected/adf7837a-7e9f-4612-a770-52b31b7a06d9-kube-api-access-wrpm7\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379161 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.379213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379196 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cqr6w\" (UniqueName: \"kubernetes.io/projected/7f0e74e3-e3bf-4668-872f-d3b519382477-kube-api-access-cqr6w\") pod \"volume-data-source-validator-7598fd5bd-g5rnm\" (UID: \"7f0e74e3-e3bf-4668-872f-d3b519382477\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" Jun 02 21:40:06.379492 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379219 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.379629 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.379589 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.383178 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.383153 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8796e9bb-846a-48ef-88be-45ac60ffc2fe-serving-cert\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.383304 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.383268 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.383304 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.383290 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.404127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.404094 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4wlt\" (UniqueName: \"kubernetes.io/projected/8796e9bb-846a-48ef-88be-45ac60ffc2fe-kube-api-access-t4wlt\") pod \"kube-storage-version-migrator-operator-96ddc787c-2swqs\" (UID: \"8796e9bb-846a-48ef-88be-45ac60ffc2fe\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.404291 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.404256 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.405063 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.405040 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqr6w\" (UniqueName: \"kubernetes.io/projected/7f0e74e3-e3bf-4668-872f-d3b519382477-kube-api-access-cqr6w\") pod \"volume-data-source-validator-7598fd5bd-g5rnm\" (UID: \"7f0e74e3-e3bf-4668-872f-d3b519382477\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" Jun 02 21:40:06.405517 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.405494 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn6t7\" (UniqueName: \"kubernetes.io/projected/ec6d065e-18dc-4ce8-9741-5dc7242d0078-kube-api-access-gn6t7\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.405618 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.405575 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4ljz\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.450633 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.450544 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" Jun 02 21:40:06.467809 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.467772 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" Jun 02 21:40:06.479891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.479853 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrpm7\" (UniqueName: \"kubernetes.io/projected/adf7837a-7e9f-4612-a770-52b31b7a06d9-kube-api-access-wrpm7\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.479891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.479902 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.480143 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.479939 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rrzk2\" (UniqueName: \"kubernetes.io/projected/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-kube-api-access-rrzk2\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.480143 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480107 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-trusted-ca\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.480250 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480151 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qm6w6\" (UniqueName: \"kubernetes.io/projected/628721dd-9b4c-4673-9540-7457f195b066-kube-api-access-qm6w6\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.480250 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480185 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.480250 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480214 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvcd5\" (UniqueName: \"kubernetes.io/projected/d02a7b59-6742-4b76-8286-e8b53f9df26d-kube-api-access-jvcd5\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.480387 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480248 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.480387 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480307 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.480387 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480334 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64c54cf9-214c-4a6e-bddb-cfc67303da34-config-volume\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.480387 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480361 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6db5b82-1ac0-4218-be83-e1914e27f39e-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.480571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480427 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/adf7837a-7e9f-4612-a770-52b31b7a06d9-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.480571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480460 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79wkn\" (UniqueName: \"kubernetes.io/projected/64c54cf9-214c-4a6e-bddb-cfc67303da34-kube-api-access-79wkn\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.480571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480516 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6db5b82-1ac0-4218-be83-e1914e27f39e-tmp\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.480571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480549 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64c54cf9-214c-4a6e-bddb-cfc67303da34-tmp-dir\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.480740 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480592 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/658df708-29d1-44e2-83d5-33e5316847f7-serving-cert\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.480740 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480651 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zz5\" (UniqueName: \"kubernetes.io/projected/e6db5b82-1ac0-4218-be83-e1914e27f39e-kube-api-access-s2zz5\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.480740 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480678 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.480740 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480725 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ckrsr\" (UniqueName: \"kubernetes.io/projected/5c59e95b-931d-4ff3-96a5-f1f06f21a168-kube-api-access-ckrsr\") pod \"network-check-source-56d84b9f44-k7m6k\" (UID: \"5c59e95b-931d-4ff3-96a5-f1f06f21a168\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.480983 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/628721dd-9b4c-4673-9540-7457f195b066-serving-cert\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481049 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481077 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481102 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/628721dd-9b4c-4673-9540-7457f195b066-config\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481106 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/adf7837a-7e9f-4612-a770-52b31b7a06d9-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481134 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-config\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481372 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fkgc8\" (UniqueName: \"kubernetes.io/projected/658df708-29d1-44e2-83d5-33e5316847f7-kube-api-access-fkgc8\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481007 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-trusted-ca\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.481730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.481674 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/628721dd-9b4c-4673-9540-7457f195b066-config\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.482248 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.482066 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/658df708-29d1-44e2-83d5-33e5316847f7-config\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.483549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.483296 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.483945 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.483726 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-ca\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.484152 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.484134 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/628721dd-9b4c-4673-9540-7457f195b066-serving-cert\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.484219 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.484185 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.484404 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.484384 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.484753 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.484731 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/658df708-29d1-44e2-83d5-33e5316847f7-serving-cert\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.485703 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.485677 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/adf7837a-7e9f-4612-a770-52b31b7a06d9-hub\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.489710 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.489683 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrpm7\" (UniqueName: \"kubernetes.io/projected/adf7837a-7e9f-4612-a770-52b31b7a06d9-kube-api-access-wrpm7\") pod \"cluster-proxy-proxy-agent-7fcf49b977-v56q9\" (UID: \"adf7837a-7e9f-4612-a770-52b31b7a06d9\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.489906 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.489885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm6w6\" (UniqueName: \"kubernetes.io/projected/628721dd-9b4c-4673-9540-7457f195b066-kube-api-access-qm6w6\") pod \"service-ca-operator-69877444c-2n4nc\" (UID: \"628721dd-9b4c-4673-9540-7457f195b066\") " pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.490044 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.490015 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckrsr\" (UniqueName: \"kubernetes.io/projected/5c59e95b-931d-4ff3-96a5-f1f06f21a168-kube-api-access-ckrsr\") pod \"network-check-source-56d84b9f44-k7m6k\" (UID: \"5c59e95b-931d-4ff3-96a5-f1f06f21a168\") " pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" Jun 02 21:40:06.490196 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.490152 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrzk2\" (UniqueName: \"kubernetes.io/projected/8c6a860d-ef9a-4188-b313-5df25ec4f6bc-kube-api-access-rrzk2\") pod \"managed-serviceaccount-addon-agent-54f7957f84-p6hb8\" (UID: \"8c6a860d-ef9a-4188-b313-5df25ec4f6bc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.492069 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.492027 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkgc8\" (UniqueName: \"kubernetes.io/projected/658df708-29d1-44e2-83d5-33e5316847f7-kube-api-access-fkgc8\") pod \"console-operator-86d7c6cc77-88rls\" (UID: \"658df708-29d1-44e2-83d5-33e5316847f7\") " pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.516549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.516512 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" Jun 02 21:40:06.534427 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.534389 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:06.566211 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.566171 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" Jun 02 21:40:06.575070 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.575041 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:40:06.582021 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.581979 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64c54cf9-214c-4a6e-bddb-cfc67303da34-tmp-dir\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582148 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582077 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zz5\" (UniqueName: \"kubernetes.io/projected/e6db5b82-1ac0-4218-be83-e1914e27f39e-kube-api-access-s2zz5\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.582148 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582139 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.582249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582159 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jvcd5\" (UniqueName: \"kubernetes.io/projected/d02a7b59-6742-4b76-8286-e8b53f9df26d-kube-api-access-jvcd5\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.582249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582180 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582203 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64c54cf9-214c-4a6e-bddb-cfc67303da34-config-volume\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582231 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6db5b82-1ac0-4218-be83-e1914e27f39e-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.582426 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.582253 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:06.582426 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.582331 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:06.582426 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582364 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64c54cf9-214c-4a6e-bddb-cfc67303da34-tmp-dir\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582426 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582261 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-79wkn\" (UniqueName: \"kubernetes.io/projected/64c54cf9-214c-4a6e-bddb-cfc67303da34-kube-api-access-79wkn\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582656 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.582335 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:07.082313087 +0000 UTC m=+34.883674053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:06.582656 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.582520 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:07.082505711 +0000 UTC m=+34.883866664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:06.582656 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582556 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6db5b82-1ac0-4218-be83-e1914e27f39e-tmp\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.582784 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582766 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64c54cf9-214c-4a6e-bddb-cfc67303da34-config-volume\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.582908 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.582885 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6db5b82-1ac0-4218-be83-e1914e27f39e-tmp\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.585074 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.585055 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/e6db5b82-1ac0-4218-be83-e1914e27f39e-klusterlet-config\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.591142 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.591122 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zz5\" (UniqueName: \"kubernetes.io/projected/e6db5b82-1ac0-4218-be83-e1914e27f39e-kube-api-access-s2zz5\") pod \"klusterlet-addon-workmgr-69b5b8dc75-d84f6\" (UID: \"e6db5b82-1ac0-4218-be83-e1914e27f39e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.595524 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.595495 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvcd5\" (UniqueName: \"kubernetes.io/projected/d02a7b59-6742-4b76-8286-e8b53f9df26d-kube-api-access-jvcd5\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:06.596276 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.596254 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79wkn\" (UniqueName: \"kubernetes.io/projected/64c54cf9-214c-4a6e-bddb-cfc67303da34-kube-api-access-79wkn\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:06.602907 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.602885 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" Jun 02 21:40:06.616777 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.616737 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:06.806057 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.805954 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:06.806057 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.805979 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:06.806057 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.805954 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:06.810119 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.810092 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 21:40:06.810261 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.810140 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qhklz\"" Jun 02 21:40:06.811160 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.811134 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:40:06.812098 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.812076 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8znwf\"" Jun 02 21:40:06.885610 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.885579 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:06.885797 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:06.885633 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:06.885797 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.885767 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:06.885797 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.885787 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:06.885938 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.885798 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:06.885938 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.885859 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:07.88584284 +0000 UTC m=+35.687203796 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:06.885938 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:06.885876 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:07.88586749 +0000 UTC m=+35.687228453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:07.086948 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:07.086861 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:07.086948 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:07.086911 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:07.087442 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.087054 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:07.087442 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.087104 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:07.087442 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.087130 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:08.087111428 +0000 UTC m=+35.888472387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:07.087442 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.087206 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:08.087187847 +0000 UTC m=+35.888548815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:07.893653 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:07.893607 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:07.893852 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.893769 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:07.893922 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.893902 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:07.894162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:07.894133 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:07.894300 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.894168 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:07.894598 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.894577 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:09.894550974 +0000 UTC m=+37.695911943 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:07.897478 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:07.894699 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:09.894666091 +0000 UTC m=+37.696027055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:08.096516 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.096467 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:08.097039 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.096530 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:08.097039 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.096660 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:08.097039 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.096681 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:08.097039 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.096753 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:10.096734246 +0000 UTC m=+37.898095221 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:08.097039 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.096773 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:10.096763785 +0000 UTC m=+37.898124741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:08.563569 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.562607 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k"] Jun 02 21:40:08.566397 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.565668 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm"] Jun 02 21:40:08.574490 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.574455 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c59e95b_931d_4ff3_96a5_f1f06f21a168.slice/crio-c510e531c7ace52347f787f9494e732ae98aec78629099c2009691206fc0aa7d WatchSource:0}: Error finding container c510e531c7ace52347f787f9494e732ae98aec78629099c2009691206fc0aa7d: Status 404 returned error can't find the container with id c510e531c7ace52347f787f9494e732ae98aec78629099c2009691206fc0aa7d Jun 02 21:40:08.577460 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.577436 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f0e74e3_e3bf_4668_872f_d3b519382477.slice/crio-d021e5823bc4bc1b571f7939375fd507c4e463347baafcb3d32190be448b13b8 WatchSource:0}: Error finding container d021e5823bc4bc1b571f7939375fd507c4e463347baafcb3d32190be448b13b8: Status 404 returned error can't find the container with id d021e5823bc4bc1b571f7939375fd507c4e463347baafcb3d32190be448b13b8 Jun 02 21:40:08.594284 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.594241 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8"] Jun 02 21:40:08.595471 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.595417 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9"] Jun 02 21:40:08.597779 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.597228 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc"] Jun 02 21:40:08.598727 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.598705 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs"] Jun 02 21:40:08.600069 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.599465 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628721dd_9b4c_4673_9540_7457f195b066.slice/crio-2ec2fabccfb0ce1ec69390ea2b5c3bbbdd247b568fcc9b3b5054067a9f016b8c WatchSource:0}: Error finding container 2ec2fabccfb0ce1ec69390ea2b5c3bbbdd247b568fcc9b3b5054067a9f016b8c: Status 404 returned error can't find the container with id 2ec2fabccfb0ce1ec69390ea2b5c3bbbdd247b568fcc9b3b5054067a9f016b8c Jun 02 21:40:08.600069 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.599670 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c6a860d_ef9a_4188_b313_5df25ec4f6bc.slice/crio-56d41ac9b0825d9e47ea048a09282c854d34fc8aef5b061bff90dcfa3d6b15a7 WatchSource:0}: Error finding container 56d41ac9b0825d9e47ea048a09282c854d34fc8aef5b061bff90dcfa3d6b15a7: Status 404 returned error can't find the container with id 56d41ac9b0825d9e47ea048a09282c854d34fc8aef5b061bff90dcfa3d6b15a7 Jun 02 21:40:08.600069 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.599829 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-86d7c6cc77-88rls"] Jun 02 21:40:08.600520 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.600415 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadf7837a_7e9f_4612_a770_52b31b7a06d9.slice/crio-13a472fa98dbf2808e1d6617c194cb4fceecb55a10948ba5bd7642cfa8aa2fa4 WatchSource:0}: Error finding container 13a472fa98dbf2808e1d6617c194cb4fceecb55a10948ba5bd7642cfa8aa2fa4: Status 404 returned error can't find the container with id 13a472fa98dbf2808e1d6617c194cb4fceecb55a10948ba5bd7642cfa8aa2fa4 Jun 02 21:40:08.601444 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.601101 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6"] Jun 02 21:40:08.602358 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.601930 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv"] Jun 02 21:40:08.603431 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.603157 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658df708_29d1_44e2_83d5_33e5316847f7.slice/crio-60adfc311dc2fd05e07bc00acd5d5e60dc6e29736fc454550d8406feb21a8c09 WatchSource:0}: Error finding container 60adfc311dc2fd05e07bc00acd5d5e60dc6e29736fc454550d8406feb21a8c09: Status 404 returned error can't find the container with id 60adfc311dc2fd05e07bc00acd5d5e60dc6e29736fc454550d8406feb21a8c09 Jun 02 21:40:08.605251 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.604929 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8796e9bb_846a_48ef_88be_45ac60ffc2fe.slice/crio-e56fb43a343fe965147c3b4c8c0925b6997f7c772b45bd043a164ce7f663d7dd WatchSource:0}: Error finding container e56fb43a343fe965147c3b4c8c0925b6997f7c772b45bd043a164ce7f663d7dd: Status 404 returned error can't find the container with id e56fb43a343fe965147c3b4c8c0925b6997f7c772b45bd043a164ce7f663d7dd Jun 02 21:40:08.605442 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:08.605423 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6db5b82_1ac0_4218_be83_e1914e27f39e.slice/crio-d60525e925795c27b96a76466de5d5f82b089a47078041dfdfdcbbc6160bf356 WatchSource:0}: Error finding container d60525e925795c27b96a76466de5d5f82b089a47078041dfdfdcbbc6160bf356: Status 404 returned error can't find the container with id d60525e925795c27b96a76466de5d5f82b089a47078041dfdfdcbbc6160bf356 Jun 02 21:40:08.627914 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.627881 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv"] Jun 02 21:40:08.628068 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.628054 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.631032 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.630749 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 02 21:40:08.631032 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.630761 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4n9hw\"" Jun 02 21:40:08.631032 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.630804 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 02 21:40:08.803525 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.803331 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.803685 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.803561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/775cb3c0-90b1-4201-b45d-9255bb856130-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.904970 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.904931 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.905120 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.905014 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/775cb3c0-90b1-4201-b45d-9255bb856130-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.905171 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.905108 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:08.905220 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:08.905188 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:09.405172111 +0000 UTC m=+37.206533068 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:08.905668 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.905648 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/775cb3c0-90b1-4201-b45d-9255bb856130-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:08.994403 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.994311 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="2543737404ad2e7f364860ba5fcefe54b5d2c6bd3d215c9309f4ce0332361daa" exitCode=0 Jun 02 21:40:08.994549 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.994393 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"2543737404ad2e7f364860ba5fcefe54b5d2c6bd3d215c9309f4ce0332361daa"} Jun 02 21:40:08.995582 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.995556 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" event={"ID":"8796e9bb-846a-48ef-88be-45ac60ffc2fe","Type":"ContainerStarted","Data":"e56fb43a343fe965147c3b4c8c0925b6997f7c772b45bd043a164ce7f663d7dd"} Jun 02 21:40:08.996518 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.996491 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" event={"ID":"7f0e74e3-e3bf-4668-872f-d3b519382477","Type":"ContainerStarted","Data":"d021e5823bc4bc1b571f7939375fd507c4e463347baafcb3d32190be448b13b8"} Jun 02 21:40:08.997670 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.997648 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerStarted","Data":"13a472fa98dbf2808e1d6617c194cb4fceecb55a10948ba5bd7642cfa8aa2fa4"} Jun 02 21:40:08.998724 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.998697 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" event={"ID":"628721dd-9b4c-4673-9540-7457f195b066","Type":"ContainerStarted","Data":"2ec2fabccfb0ce1ec69390ea2b5c3bbbdd247b568fcc9b3b5054067a9f016b8c"} Jun 02 21:40:08.999664 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:08.999645 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" event={"ID":"e6db5b82-1ac0-4218-be83-e1914e27f39e","Type":"ContainerStarted","Data":"d60525e925795c27b96a76466de5d5f82b089a47078041dfdfdcbbc6160bf356"} Jun 02 21:40:09.000539 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.000520 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" event={"ID":"8c6a860d-ef9a-4188-b313-5df25ec4f6bc","Type":"ContainerStarted","Data":"56d41ac9b0825d9e47ea048a09282c854d34fc8aef5b061bff90dcfa3d6b15a7"} Jun 02 21:40:09.001414 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.001395 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" event={"ID":"658df708-29d1-44e2-83d5-33e5316847f7","Type":"ContainerStarted","Data":"60adfc311dc2fd05e07bc00acd5d5e60dc6e29736fc454550d8406feb21a8c09"} Jun 02 21:40:09.002177 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.002159 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" event={"ID":"5c59e95b-931d-4ff3-96a5-f1f06f21a168","Type":"ContainerStarted","Data":"c510e531c7ace52347f787f9494e732ae98aec78629099c2009691206fc0aa7d"} Jun 02 21:40:09.410381 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.410337 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:09.411278 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.410663 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:09.411278 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.410733 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:10.410713881 +0000 UTC m=+38.212074849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:09.917715 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.917673 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:09.917891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:09.917739 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:09.917968 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.917942 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:09.918048 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.918024 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:13.918003913 +0000 UTC m=+41.719364880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:09.918457 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.918435 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:09.918457 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.918459 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:09.918677 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:09.918506 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:13.918490836 +0000 UTC m=+41.719851804 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:10.029845 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:10.028927 2558 generic.go:358] "Generic (PLEG): container finished" podID="744ab60f-c16e-4f34-8d7a-879cde111ece" containerID="df1968c7aebba5733895c3a3cd53591022f4004ca6461d009dc65d7ca3c60b4a" exitCode=0 Jun 02 21:40:10.029845 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:10.028977 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerDied","Data":"df1968c7aebba5733895c3a3cd53591022f4004ca6461d009dc65d7ca3c60b4a"} Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:10.119338 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:10.119403 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.119493 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.119569 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:14.119547018 +0000 UTC m=+41.920907976 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.119619 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:10.119716 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.119669 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:14.1196538 +0000 UTC m=+41.921014767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:10.423379 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:10.422688 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:10.423379 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.422904 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:10.423379 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:10.422973 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:12.422953474 +0000 UTC m=+40.224314445 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:11.108059 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:11.107952 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" event={"ID":"744ab60f-c16e-4f34-8d7a-879cde111ece","Type":"ContainerStarted","Data":"ee85cade2c794749c82d4a5984b49e136863f063ee28c4e2d49fa9f5cd2eb806"} Jun 02 21:40:11.138983 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:11.138416 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2k9mn" podStartSLOduration=4.588790925 podStartE2EDuration="39.138396293s" podCreationTimestamp="2026-06-02 21:39:32 +0000 UTC" firstStartedPulling="2026-06-02 21:39:34.038724038 +0000 UTC m=+1.840084991" lastFinishedPulling="2026-06-02 21:40:08.588329403 +0000 UTC m=+36.389690359" observedRunningTime="2026-06-02 21:40:11.136341219 +0000 UTC m=+38.937702195" watchObservedRunningTime="2026-06-02 21:40:11.138396293 +0000 UTC m=+38.939757269" Jun 02 21:40:12.445252 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:12.445207 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:12.445734 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:12.445376 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:12.445734 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:12.445462 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:16.445440908 +0000 UTC m=+44.246801868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:13.959190 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:13.959149 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:13.959212 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:13.959324 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:13.959349 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:13.959424 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:21.959403763 +0000 UTC m=+49.760764726 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:13.959351 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:13.959767 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:13.959531 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:21.959511359 +0000 UTC m=+49.760872325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:14.160130 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:14.160090 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:14.160130 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:14.160130 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:14.160331 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:14.160208 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:14.160331 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:14.160274 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:22.160258189 +0000 UTC m=+49.961619147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:14.160331 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:14.160278 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:14.160434 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:14.160329 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:22.16031711 +0000 UTC m=+49.961678068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:16.479922 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:16.479867 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:16.480418 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:16.480073 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:16.480418 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:16.480166 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:24.480141462 +0000 UTC m=+52.281502434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:16.681025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:16.680963 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:16.684911 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:16.684878 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e6fcc949-f931-481e-8bce-c896aaaa5189-original-pull-secret\") pod \"global-pull-secret-syncer-tpm87\" (UID: \"e6fcc949-f931-481e-8bce-c896aaaa5189\") " pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:16.726107 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:16.726072 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tpm87" Jun 02 21:40:21.559404 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:21.559311 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tpm87"] Jun 02 21:40:21.562196 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:21.562169 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6fcc949_f931_481e_8bce_c896aaaa5189.slice/crio-57bc3626686a4a7d37010b31b7e4e9bf7ae11a5713c3833a4b14accdc84dcb99 WatchSource:0}: Error finding container 57bc3626686a4a7d37010b31b7e4e9bf7ae11a5713c3833a4b14accdc84dcb99: Status 404 returned error can't find the container with id 57bc3626686a4a7d37010b31b7e4e9bf7ae11a5713c3833a4b14accdc84dcb99 Jun 02 21:40:22.024847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.024796 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:22.025084 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.024856 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:22.025084 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.025003 2558 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:40:22.025084 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.025036 2558 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-58c9c4d9d-r7d8x: secret "image-registry-tls" not found Jun 02 21:40:22.025249 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.024983 2558 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 02 21:40:22.025249 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.025124 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls podName:17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:38.025104194 +0000 UTC m=+65.826465152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls") pod "image-registry-58c9c4d9d-r7d8x" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46") : secret "image-registry-tls" not found Jun 02 21:40:22.025249 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.025175 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls podName:ec6d065e-18dc-4ce8-9741-5dc7242d0078 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:38.025152889 +0000 UTC m=+65.826513857 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls") pod "cluster-samples-operator-755bbf9bfb-5ggxv" (UID: "ec6d065e-18dc-4ce8-9741-5dc7242d0078") : secret "samples-operator-tls" not found Jun 02 21:40:22.137479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.137047 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" event={"ID":"8796e9bb-846a-48ef-88be-45ac60ffc2fe","Type":"ContainerStarted","Data":"22491f55ebc5a69d5576de0addd5a7c154f07918595ebe19f772ac7d3015d821"} Jun 02 21:40:22.139018 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.138768 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" event={"ID":"7f0e74e3-e3bf-4668-872f-d3b519382477","Type":"ContainerStarted","Data":"dac517664966299a7b3efb6dc512347e3576b3c62f3dafcaf00339b095f24276"} Jun 02 21:40:22.140564 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.140376 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerStarted","Data":"6e5157ae40b19cbff812b476b9ea80be6a2f29b12505a4dc92d897acf3554e61"} Jun 02 21:40:22.142360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.142323 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" event={"ID":"628721dd-9b4c-4673-9540-7457f195b066","Type":"ContainerStarted","Data":"bc85590def886e1d1a9f03910d3b064b3ead14d87cc32e901b08c2995dbb6c0f"} Jun 02 21:40:22.145414 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.145152 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" event={"ID":"e6db5b82-1ac0-4218-be83-e1914e27f39e","Type":"ContainerStarted","Data":"d6e1935642dfddd5860fc45971e1cea0e2d93390a4c76a7610cb46bcf7879081"} Jun 02 21:40:22.145532 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.145430 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:22.147228 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.147206 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" Jun 02 21:40:22.147675 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.147644 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" event={"ID":"8c6a860d-ef9a-4188-b313-5df25ec4f6bc","Type":"ContainerStarted","Data":"2e9eb83d5fdbbaca5d95bed3f658e3deaaa3dcf0cd6401dfd9f20e26077b9e06"} Jun 02 21:40:22.149805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.149642 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/0.log" Jun 02 21:40:22.149805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.149683 2558 generic.go:358] "Generic (PLEG): container finished" podID="658df708-29d1-44e2-83d5-33e5316847f7" containerID="bc3595b09d770a3d735694f85e20951e4cf8bc9814372d8f1053ea0e382cc483" exitCode=255 Jun 02 21:40:22.149805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.149765 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" event={"ID":"658df708-29d1-44e2-83d5-33e5316847f7","Type":"ContainerDied","Data":"bc3595b09d770a3d735694f85e20951e4cf8bc9814372d8f1053ea0e382cc483"} Jun 02 21:40:22.150039 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.149953 2558 scope.go:117] "RemoveContainer" containerID="bc3595b09d770a3d735694f85e20951e4cf8bc9814372d8f1053ea0e382cc483" Jun 02 21:40:22.152548 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.152423 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" event={"ID":"5c59e95b-931d-4ff3-96a5-f1f06f21a168","Type":"ContainerStarted","Data":"3fa74e80934b009901e9b3c4e3669fbeedd4c4d362fea0a28d84acb416040c29"} Jun 02 21:40:22.154124 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.154090 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tpm87" event={"ID":"e6fcc949-f931-481e-8bce-c896aaaa5189","Type":"ContainerStarted","Data":"57bc3626686a4a7d37010b31b7e4e9bf7ae11a5713c3833a4b14accdc84dcb99"} Jun 02 21:40:22.158062 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.158014 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" podStartSLOduration=35.339428488 podStartE2EDuration="48.157978196s" podCreationTimestamp="2026-06-02 21:39:34 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.607382587 +0000 UTC m=+36.408743550" lastFinishedPulling="2026-06-02 21:40:21.425932291 +0000 UTC m=+49.227293258" observedRunningTime="2026-06-02 21:40:22.157108368 +0000 UTC m=+49.958469341" watchObservedRunningTime="2026-06-02 21:40:22.157978196 +0000 UTC m=+49.959339171" Jun 02 21:40:22.213975 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.213461 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-69b5b8dc75-d84f6" podStartSLOduration=26.373378796 podStartE2EDuration="39.213441387s" podCreationTimestamp="2026-06-02 21:39:43 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.607734019 +0000 UTC m=+36.409094975" lastFinishedPulling="2026-06-02 21:40:21.447796613 +0000 UTC m=+49.249157566" observedRunningTime="2026-06-02 21:40:22.189701761 +0000 UTC m=+49.991062739" watchObservedRunningTime="2026-06-02 21:40:22.213441387 +0000 UTC m=+50.014802363" Jun 02 21:40:22.226949 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.226918 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:22.227127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.227042 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:22.228099 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.227719 2558 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:40:22.228099 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.227778 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert podName:d02a7b59-6742-4b76-8286-e8b53f9df26d nodeName:}" failed. No retries permitted until 2026-06-02 21:40:38.227759185 +0000 UTC m=+66.029120137 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert") pod "ingress-canary-4pbgs" (UID: "d02a7b59-6742-4b76-8286-e8b53f9df26d") : secret "canary-serving-cert" not found Jun 02 21:40:22.234045 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.230675 2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:40:22.234045 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:22.230738 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls podName:64c54cf9-214c-4a6e-bddb-cfc67303da34 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:38.230720474 +0000 UTC m=+66.032081430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls") pod "dns-default-qhxs5" (UID: "64c54cf9-214c-4a6e-bddb-cfc67303da34") : secret "dns-default-metrics-tls" not found Jun 02 21:40:22.255308 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.255092 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" podStartSLOduration=35.430679059 podStartE2EDuration="48.255072235s" podCreationTimestamp="2026-06-02 21:39:34 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.600889383 +0000 UTC m=+36.402250339" lastFinishedPulling="2026-06-02 21:40:21.425282556 +0000 UTC m=+49.226643515" observedRunningTime="2026-06-02 21:40:22.214296518 +0000 UTC m=+50.015657474" watchObservedRunningTime="2026-06-02 21:40:22.255072235 +0000 UTC m=+50.056433213" Jun 02 21:40:22.285317 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.285077 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-54f7957f84-p6hb8" podStartSLOduration=26.462651219 podStartE2EDuration="39.285058152s" podCreationTimestamp="2026-06-02 21:39:43 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.602881026 +0000 UTC m=+36.404241987" lastFinishedPulling="2026-06-02 21:40:21.425287963 +0000 UTC m=+49.226648920" observedRunningTime="2026-06-02 21:40:22.28426794 +0000 UTC m=+50.085628916" watchObservedRunningTime="2026-06-02 21:40:22.285058152 +0000 UTC m=+50.086419128" Jun 02 21:40:22.302344 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.302293 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56d84b9f44-k7m6k" podStartSLOduration=25.451695866 podStartE2EDuration="38.302277478s" podCreationTimestamp="2026-06-02 21:39:44 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.576742158 +0000 UTC m=+36.378103114" lastFinishedPulling="2026-06-02 21:40:21.42732376 +0000 UTC m=+49.228684726" observedRunningTime="2026-06-02 21:40:22.301244653 +0000 UTC m=+50.102605629" watchObservedRunningTime="2026-06-02 21:40:22.302277478 +0000 UTC m=+50.103638453" Jun 02 21:40:22.323227 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:22.323169 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-7598fd5bd-g5rnm" podStartSLOduration=37.455258281 podStartE2EDuration="48.323128998s" podCreationTimestamp="2026-06-02 21:39:34 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.579617615 +0000 UTC m=+36.380978567" lastFinishedPulling="2026-06-02 21:40:19.447488315 +0000 UTC m=+47.248849284" observedRunningTime="2026-06-02 21:40:22.319969382 +0000 UTC m=+50.121330359" watchObservedRunningTime="2026-06-02 21:40:22.323128998 +0000 UTC m=+50.124489974" Jun 02 21:40:23.159269 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.159219 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.159980 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/0.log" Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.160036 2558 generic.go:358] "Generic (PLEG): container finished" podID="658df708-29d1-44e2-83d5-33e5316847f7" containerID="5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54" exitCode=255 Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.160158 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" event={"ID":"658df708-29d1-44e2-83d5-33e5316847f7","Type":"ContainerDied","Data":"5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54"} Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.160208 2558 scope.go:117] "RemoveContainer" containerID="bc3595b09d770a3d735694f85e20951e4cf8bc9814372d8f1053ea0e382cc483" Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:23.160395 2558 scope.go:117] "RemoveContainer" containerID="5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54" Jun 02 21:40:23.160636 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:23.160600 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-88rls_openshift-console-operator(658df708-29d1-44e2-83d5-33e5316847f7)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" podUID="658df708-29d1-44e2-83d5-33e5316847f7" Jun 02 21:40:24.164379 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:24.164347 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:40:24.164817 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:24.164780 2558 scope.go:117] "RemoveContainer" containerID="5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54" Jun 02 21:40:24.165067 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:24.165042 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-88rls_openshift-console-operator(658df708-29d1-44e2-83d5-33e5316847f7)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" podUID="658df708-29d1-44e2-83d5-33e5316847f7" Jun 02 21:40:24.552066 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:24.551954 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:24.552201 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:24.552114 2558 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:40:24.552201 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:24.552186 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert podName:775cb3c0-90b1-4201-b45d-9255bb856130 nodeName:}" failed. No retries permitted until 2026-06-02 21:40:40.552169249 +0000 UTC m=+68.353530202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-5nlzv" (UID: "775cb3c0-90b1-4201-b45d-9255bb856130") : secret "networking-console-plugin-cert" not found Jun 02 21:40:25.169254 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:25.169215 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerStarted","Data":"14078beffe5b28b6fdfb5cda0d5ad93270a63f115ace5933d0af444534b5f65e"} Jun 02 21:40:25.169254 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:25.169258 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerStarted","Data":"75e8c8c78658cc34b0bec28ec8b49a5f378d106a449d6cacef0453f601ca9185"} Jun 02 21:40:25.191919 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:25.191861 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" podStartSLOduration=25.922056348 podStartE2EDuration="42.191844584s" podCreationTimestamp="2026-06-02 21:39:43 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.603517632 +0000 UTC m=+36.404878586" lastFinishedPulling="2026-06-02 21:40:24.873305854 +0000 UTC m=+52.674666822" observedRunningTime="2026-06-02 21:40:25.190124945 +0000 UTC m=+52.991485919" watchObservedRunningTime="2026-06-02 21:40:25.191844584 +0000 UTC m=+52.993205559" Jun 02 21:40:26.534667 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:26.534624 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:26.534667 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:26.534674 2558 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:26.535192 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:26.535162 2558 scope.go:117] "RemoveContainer" containerID="5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54" Jun 02 21:40:26.535411 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:26.535388 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-86d7c6cc77-88rls_openshift-console-operator(658df708-29d1-44e2-83d5-33e5316847f7)\"" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" podUID="658df708-29d1-44e2-83d5-33e5316847f7" Jun 02 21:40:27.176891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:27.176850 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tpm87" event={"ID":"e6fcc949-f931-481e-8bce-c896aaaa5189","Type":"ContainerStarted","Data":"34a545dc2537275e1c214bf308ee0c936f8e16a3b6acffec46be6f5dfbc70ced"} Jun 02 21:40:27.204705 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:27.204646 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tpm87" podStartSLOduration=37.788804599 podStartE2EDuration="43.204627053s" podCreationTimestamp="2026-06-02 21:39:44 +0000 UTC" firstStartedPulling="2026-06-02 21:40:21.566315557 +0000 UTC m=+49.367676511" lastFinishedPulling="2026-06-02 21:40:26.982138009 +0000 UTC m=+54.783498965" observedRunningTime="2026-06-02 21:40:27.204422175 +0000 UTC m=+55.005783149" watchObservedRunningTime="2026-06-02 21:40:27.204627053 +0000 UTC m=+55.005988027" Jun 02 21:40:32.994691 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:32.994656 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rwwhv" Jun 02 21:40:37.569150 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.569056 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:40:37.571638 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.571616 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:40:37.579892 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:37.579872 2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:40:37.579950 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:40:37.579934 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs podName:afdad777-7975-4eef-bc20-ef9878174b53 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:41.579917215 +0000 UTC m=+129.381278168 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs") pod "network-metrics-daemon-s6gxd" (UID: "afdad777-7975-4eef-bc20-ef9878174b53") : secret "metrics-daemon-secret" not found Jun 02 21:40:37.669985 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.669952 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:37.672464 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.672441 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlhdx\" (UniqueName: \"kubernetes.io/projected/279d6e77-7104-49f6-b3a4-df29f2bb64e2-kube-api-access-xlhdx\") pod \"network-check-target-4q595\" (UID: \"279d6e77-7104-49f6-b3a4-df29f2bb64e2\") " pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:37.720861 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.720821 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qhklz\"" Jun 02 21:40:37.729128 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.729103 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:37.806679 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.806651 2558 scope.go:117] "RemoveContainer" containerID="5470bb62bb5a06156adcf78b8660c7700019f34709c475f16ac65f5aa9655d54" Jun 02 21:40:37.858393 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:37.858362 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4q595"] Jun 02 21:40:37.862362 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:37.862335 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod279d6e77_7104_49f6_b3a4_df29f2bb64e2.slice/crio-b0cc6c8438f91f0dc2cf731654f9541a8680f64312895bc824a9332b0c526206 WatchSource:0}: Error finding container b0cc6c8438f91f0dc2cf731654f9541a8680f64312895bc824a9332b0c526206: Status 404 returned error can't find the container with id b0cc6c8438f91f0dc2cf731654f9541a8680f64312895bc824a9332b0c526206 Jun 02 21:40:38.074114 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.074008 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:38.074114 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.074062 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:38.076605 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.076569 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ec6d065e-18dc-4ce8-9741-5dc7242d0078-samples-operator-tls\") pod \"cluster-samples-operator-755bbf9bfb-5ggxv\" (UID: \"ec6d065e-18dc-4ce8-9741-5dc7242d0078\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:38.076715 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.076606 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"image-registry-58c9c4d9d-r7d8x\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:38.210935 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.210906 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:40:38.211148 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.211036 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" event={"ID":"658df708-29d1-44e2-83d5-33e5316847f7","Type":"ContainerStarted","Data":"4fbbe44b1b89f923bbe2e355c99f97f20477163f84e4d56a817679287a3c24d3"} Jun 02 21:40:38.211433 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.211398 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:38.212422 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.212387 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4q595" event={"ID":"279d6e77-7104-49f6-b3a4-df29f2bb64e2","Type":"ContainerStarted","Data":"de463bc489b29c1e0c047b97fc2eb801c5858a9b6cad739fa55bc1968632cce3"} Jun 02 21:40:38.212553 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.212428 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4q595" event={"ID":"279d6e77-7104-49f6-b3a4-df29f2bb64e2","Type":"ContainerStarted","Data":"b0cc6c8438f91f0dc2cf731654f9541a8680f64312895bc824a9332b0c526206"} Jun 02 21:40:38.212553 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.212532 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:40:38.228060 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.228005 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" podStartSLOduration=51.407012766 podStartE2EDuration="1m4.227974053s" podCreationTimestamp="2026-06-02 21:39:34 +0000 UTC" firstStartedPulling="2026-06-02 21:40:08.604367755 +0000 UTC m=+36.405728710" lastFinishedPulling="2026-06-02 21:40:21.425329033 +0000 UTC m=+49.226689997" observedRunningTime="2026-06-02 21:40:38.227776028 +0000 UTC m=+66.029136996" watchObservedRunningTime="2026-06-02 21:40:38.227974053 +0000 UTC m=+66.029335028" Jun 02 21:40:38.236944 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.236922 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrw4w\"" Jun 02 21:40:38.242410 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.242365 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4q595" podStartSLOduration=66.242350302 podStartE2EDuration="1m6.242350302s" podCreationTimestamp="2026-06-02 21:39:32 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:40:38.242178446 +0000 UTC m=+66.043539421" watchObservedRunningTime="2026-06-02 21:40:38.242350302 +0000 UTC m=+66.043711278" Jun 02 21:40:38.245301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.245282 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:38.276233 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.276187 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:38.276395 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.276239 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:38.278828 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.278795 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d02a7b59-6742-4b76-8286-e8b53f9df26d-cert\") pod \"ingress-canary-4pbgs\" (UID: \"d02a7b59-6742-4b76-8286-e8b53f9df26d\") " pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:38.278958 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.278893 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64c54cf9-214c-4a6e-bddb-cfc67303da34-metrics-tls\") pod \"dns-default-qhxs5\" (UID: \"64c54cf9-214c-4a6e-bddb-cfc67303da34\") " pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:38.299593 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.299324 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7c72w\"" Jun 02 21:40:38.306533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.306510 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" Jun 02 21:40:38.381095 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.381060 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:40:38.387454 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:38.387426 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17784d2f_8b5f_4eb9_a21d_1b59a8ca3e46.slice/crio-b814129808b10a0e543ff1935b9765380507ddfca32450efd5cc07b6e25dee46 WatchSource:0}: Error finding container b814129808b10a0e543ff1935b9765380507ddfca32450efd5cc07b6e25dee46: Status 404 returned error can't find the container with id b814129808b10a0e543ff1935b9765380507ddfca32450efd5cc07b6e25dee46 Jun 02 21:40:38.443285 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.443252 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2vddp\"" Jun 02 21:40:38.445065 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.445039 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv"] Jun 02 21:40:38.451573 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.451545 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:38.457343 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.457318 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vfhfl\"" Jun 02 21:40:38.463262 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.463239 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-86d7c6cc77-88rls" Jun 02 21:40:38.464980 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.464957 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4pbgs" Jun 02 21:40:38.611553 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.611432 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qhxs5"] Jun 02 21:40:38.614492 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:38.614449 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c54cf9_214c_4a6e_bddb_cfc67303da34.slice/crio-0aa7f5e77d69f54b8714114d10ab7f204fed725b749f8d9c7a49420f64dc08b7 WatchSource:0}: Error finding container 0aa7f5e77d69f54b8714114d10ab7f204fed725b749f8d9c7a49420f64dc08b7: Status 404 returned error can't find the container with id 0aa7f5e77d69f54b8714114d10ab7f204fed725b749f8d9c7a49420f64dc08b7 Jun 02 21:40:38.640768 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:38.640731 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4pbgs"] Jun 02 21:40:38.644772 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:38.644730 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd02a7b59_6742_4b76_8286_e8b53f9df26d.slice/crio-6fc209898dde1286a97c59032327d7a0bc2b5cb3be5097771b1bb26a6e3d5839 WatchSource:0}: Error finding container 6fc209898dde1286a97c59032327d7a0bc2b5cb3be5097771b1bb26a6e3d5839: Status 404 returned error can't find the container with id 6fc209898dde1286a97c59032327d7a0bc2b5cb3be5097771b1bb26a6e3d5839 Jun 02 21:40:39.218482 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.218443 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhxs5" event={"ID":"64c54cf9-214c-4a6e-bddb-cfc67303da34","Type":"ContainerStarted","Data":"0aa7f5e77d69f54b8714114d10ab7f204fed725b749f8d9c7a49420f64dc08b7"} Jun 02 21:40:39.219334 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.219310 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" event={"ID":"ec6d065e-18dc-4ce8-9741-5dc7242d0078","Type":"ContainerStarted","Data":"7b84e2fed31380cccfcef7c234a5e5134f7a34b879249658e48137f07810bd08"} Jun 02 21:40:39.220224 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.220192 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4pbgs" event={"ID":"d02a7b59-6742-4b76-8286-e8b53f9df26d","Type":"ContainerStarted","Data":"6fc209898dde1286a97c59032327d7a0bc2b5cb3be5097771b1bb26a6e3d5839"} Jun 02 21:40:39.221430 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.221406 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" event={"ID":"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46","Type":"ContainerStarted","Data":"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58"} Jun 02 21:40:39.221537 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.221437 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" event={"ID":"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46","Type":"ContainerStarted","Data":"b814129808b10a0e543ff1935b9765380507ddfca32450efd5cc07b6e25dee46"} Jun 02 21:40:39.243839 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:39.243789 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" podStartSLOduration=66.243774443 podStartE2EDuration="1m6.243774443s" podCreationTimestamp="2026-06-02 21:39:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:40:39.242810687 +0000 UTC m=+67.044171664" watchObservedRunningTime="2026-06-02 21:40:39.243774443 +0000 UTC m=+67.045135417" Jun 02 21:40:40.225658 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:40.225612 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:40:40.597558 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:40.597466 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:40.603039 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:40.602981 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/775cb3c0-90b1-4201-b45d-9255bb856130-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-5nlzv\" (UID: \"775cb3c0-90b1-4201-b45d-9255bb856130\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:40.746502 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:40.746464 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-4n9hw\"" Jun 02 21:40:40.754640 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:40.754608 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" Jun 02 21:40:42.313520 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:42.313499 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv"] Jun 02 21:40:42.317606 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:40:42.317570 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod775cb3c0_90b1_4201_b45d_9255bb856130.slice/crio-40ffcebdb17d2471c4a8eb37962814438a25fd1275dd0373f6806bbd9aaa0f14 WatchSource:0}: Error finding container 40ffcebdb17d2471c4a8eb37962814438a25fd1275dd0373f6806bbd9aaa0f14: Status 404 returned error can't find the container with id 40ffcebdb17d2471c4a8eb37962814438a25fd1275dd0373f6806bbd9aaa0f14 Jun 02 21:40:43.237389 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.237347 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhxs5" event={"ID":"64c54cf9-214c-4a6e-bddb-cfc67303da34","Type":"ContainerStarted","Data":"e5961658489c03663e898114f184c4a00b86700f5f2f6e8e8feaf67563cb510b"} Jun 02 21:40:43.237389 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.237394 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qhxs5" event={"ID":"64c54cf9-214c-4a6e-bddb-cfc67303da34","Type":"ContainerStarted","Data":"3c9f516db63e0bbf2f6c6f8d918cfc67e94c1e8d63fb5e36b122b3aa5dc35975"} Jun 02 21:40:43.237642 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.237515 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:43.239360 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.239327 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" event={"ID":"ec6d065e-18dc-4ce8-9741-5dc7242d0078","Type":"ContainerStarted","Data":"5706f8baa14c03c31a9a791dbca0b7291dc661a2ff7efd5bece4258b0c35cb7e"} Jun 02 21:40:43.239498 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.239365 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" event={"ID":"ec6d065e-18dc-4ce8-9741-5dc7242d0078","Type":"ContainerStarted","Data":"9e498f37e879ee9fabe950b2396f5aaa384b7953edc8154f4155fe21137d945e"} Jun 02 21:40:43.240789 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.240758 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4pbgs" event={"ID":"d02a7b59-6742-4b76-8286-e8b53f9df26d","Type":"ContainerStarted","Data":"8bc85f40ba88d38f1c3437275ea75b5b06a9d571043eaa77ae0cf74a79e07a61"} Jun 02 21:40:43.241965 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.241929 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" event={"ID":"775cb3c0-90b1-4201-b45d-9255bb856130","Type":"ContainerStarted","Data":"40ffcebdb17d2471c4a8eb37962814438a25fd1275dd0373f6806bbd9aaa0f14"} Jun 02 21:40:43.256289 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.256121 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-qhxs5" podStartSLOduration=33.704593583 podStartE2EDuration="37.256101462s" podCreationTimestamp="2026-06-02 21:40:06 +0000 UTC" firstStartedPulling="2026-06-02 21:40:38.616931795 +0000 UTC m=+66.418292755" lastFinishedPulling="2026-06-02 21:40:42.168439668 +0000 UTC m=+69.969800634" observedRunningTime="2026-06-02 21:40:43.25554886 +0000 UTC m=+71.056909836" watchObservedRunningTime="2026-06-02 21:40:43.256101462 +0000 UTC m=+71.057462437" Jun 02 21:40:43.274566 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.274495 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-755bbf9bfb-5ggxv" podStartSLOduration=65.624745879 podStartE2EDuration="1m9.274480457s" podCreationTimestamp="2026-06-02 21:39:34 +0000 UTC" firstStartedPulling="2026-06-02 21:40:38.518701624 +0000 UTC m=+66.320062578" lastFinishedPulling="2026-06-02 21:40:42.168436199 +0000 UTC m=+69.969797156" observedRunningTime="2026-06-02 21:40:43.273722467 +0000 UTC m=+71.075083442" watchObservedRunningTime="2026-06-02 21:40:43.274480457 +0000 UTC m=+71.075841432" Jun 02 21:40:43.290548 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:43.290499 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4pbgs" podStartSLOduration=33.766156623 podStartE2EDuration="37.290481205s" podCreationTimestamp="2026-06-02 21:40:06 +0000 UTC" firstStartedPulling="2026-06-02 21:40:38.646692652 +0000 UTC m=+66.448053604" lastFinishedPulling="2026-06-02 21:40:42.171017219 +0000 UTC m=+69.972378186" observedRunningTime="2026-06-02 21:40:43.28944081 +0000 UTC m=+71.090801785" watchObservedRunningTime="2026-06-02 21:40:43.290481205 +0000 UTC m=+71.091842180" Jun 02 21:40:44.246415 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:44.246365 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" event={"ID":"775cb3c0-90b1-4201-b45d-9255bb856130","Type":"ContainerStarted","Data":"c92de350c31b1e0a7bace29cbe8d645f906b8df24786b1a028e7ce9fc2f827e7"} Jun 02 21:40:44.263341 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:44.263292 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-5nlzv" podStartSLOduration=34.75786612 podStartE2EDuration="36.263275249s" podCreationTimestamp="2026-06-02 21:40:08 +0000 UTC" firstStartedPulling="2026-06-02 21:40:42.320215362 +0000 UTC m=+70.121576316" lastFinishedPulling="2026-06-02 21:40:43.825624487 +0000 UTC m=+71.626985445" observedRunningTime="2026-06-02 21:40:44.262867013 +0000 UTC m=+72.064227987" watchObservedRunningTime="2026-06-02 21:40:44.263275249 +0000 UTC m=+72.064636227" Jun 02 21:40:53.248652 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:53.248537 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qhxs5" Jun 02 21:40:57.284630 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:57.284595 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-5nlzv_775cb3c0-90b1-4201-b45d-9255bb856130/networking-console-plugin/0.log" Jun 02 21:40:57.432691 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:57.432657 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:40:57.630580 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:57.630452 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/2.log" Jun 02 21:40:58.249497 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:58.249453 2558 patch_prober.go:28] interesting pod/image-registry-58c9c4d9d-r7d8x container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 02 21:40:58.249668 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:58.249525 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:40:59.227376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:59.227346 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhxs5_64c54cf9-214c-4a6e-bddb-cfc67303da34/dns/0.log" Jun 02 21:40:59.427062 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:59.427032 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhxs5_64c54cf9-214c-4a6e-bddb-cfc67303da34/kube-rbac-proxy/0.log" Jun 02 21:40:59.629892 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:40:59.629818 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mqkj_037ea9d6-87f1-492a-9026-58e056b65527/dns-node-resolver/0.log" Jun 02 21:41:00.227289 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:00.227257 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58c9c4d9d-r7d8x_17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46/registry/0.log" Jun 02 21:41:00.826379 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:00.826349 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-npgt5_c7859d70-6355-4231-89d0-5b0fa8e8a79c/node-ca/0.log" Jun 02 21:41:01.232406 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:01.232378 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:41:01.426692 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:01.426661 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4pbgs_d02a7b59-6742-4b76-8286-e8b53f9df26d/serve-healthcheck-canary/0.log" Jun 02 21:41:03.556497 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.556466 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4g892"] Jun 02 21:41:03.559384 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.559360 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.563005 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.562961 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 21:41:03.564043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.563212 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bfjvd\"" Jun 02 21:41:03.564043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.563071 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 21:41:03.564043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.563017 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 21:41:03.564043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.563156 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 21:41:03.564043 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.563854 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 21:41:03.564718 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.564698 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 21:41:03.588518 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588486 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-root\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588692 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588545 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588692 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588566 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-sys\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588692 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588638 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588692 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588680 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588880 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588708 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9bvg\" (UniqueName: \"kubernetes.io/projected/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-kube-api-access-n9bvg\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588880 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588733 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-metrics-client-ca\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588880 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588817 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-textfile\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.588880 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.588850 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-wtmp\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689564 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689520 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689564 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689563 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-sys\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689586 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689610 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689640 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n9bvg\" (UniqueName: \"kubernetes.io/projected/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-kube-api-access-n9bvg\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689670 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-metrics-client-ca\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689669 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-sys\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689711 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-textfile\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689741 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-wtmp\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:41:03.689748 2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689778 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-root\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.689834 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:41:03.689819 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls podName:bcbf5c21-8561-4d56-a079-801a2e3b2eb5 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:04.189797041 +0000 UTC m=+91.991157994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls") pod "node-exporter-4g892" (UID: "bcbf5c21-8561-4d56-a079-801a2e3b2eb5") : secret "node-exporter-tls" not found Jun 02 21:41:03.690294 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689855 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-root\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.690294 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.689964 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-wtmp\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.690395 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.690371 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-textfile\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.690433 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.690398 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-metrics-client-ca\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.690467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.690434 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-accelerators-collector-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.692171 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.692150 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:03.703589 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:03.703553 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9bvg\" (UniqueName: \"kubernetes.io/projected/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-kube-api-access-n9bvg\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:04.194741 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:04.194694 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:04.197094 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:04.197058 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bcbf5c21-8561-4d56-a079-801a2e3b2eb5-node-exporter-tls\") pod \"node-exporter-4g892\" (UID: \"bcbf5c21-8561-4d56-a079-801a2e3b2eb5\") " pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:04.472312 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:04.472226 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4g892" Jun 02 21:41:04.481532 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:41:04.481498 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcbf5c21_8561_4d56_a079_801a2e3b2eb5.slice/crio-e1ff7febac1a4d46dbd634118b4fbe63c9eeaf846e20a9d052518bcc2eb43b96 WatchSource:0}: Error finding container e1ff7febac1a4d46dbd634118b4fbe63c9eeaf846e20a9d052518bcc2eb43b96: Status 404 returned error can't find the container with id e1ff7febac1a4d46dbd634118b4fbe63c9eeaf846e20a9d052518bcc2eb43b96 Jun 02 21:41:05.305948 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:05.305908 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4g892" event={"ID":"bcbf5c21-8561-4d56-a079-801a2e3b2eb5","Type":"ContainerStarted","Data":"e1ff7febac1a4d46dbd634118b4fbe63c9eeaf846e20a9d052518bcc2eb43b96"} Jun 02 21:41:06.312631 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:06.312594 2558 generic.go:358] "Generic (PLEG): container finished" podID="bcbf5c21-8561-4d56-a079-801a2e3b2eb5" containerID="b856bf3205ea0f878e59317b8d0bda1da77f9e1909380339ff2a4dcfa6d5d3da" exitCode=0 Jun 02 21:41:06.313137 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:06.312639 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4g892" event={"ID":"bcbf5c21-8561-4d56-a079-801a2e3b2eb5","Type":"ContainerDied","Data":"b856bf3205ea0f878e59317b8d0bda1da77f9e1909380339ff2a4dcfa6d5d3da"} Jun 02 21:41:07.317900 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:07.317860 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4g892" event={"ID":"bcbf5c21-8561-4d56-a079-801a2e3b2eb5","Type":"ContainerStarted","Data":"56d8fbc9a5d9751a8c09217a40fe30d30bcf61f8c13badc04a77292ab32c2707"} Jun 02 21:41:07.317900 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:07.317904 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4g892" event={"ID":"bcbf5c21-8561-4d56-a079-801a2e3b2eb5","Type":"ContainerStarted","Data":"7959a6d12adfddc1ab621be2de1286453c40a213fd429279bd9b9df97fc1235b"} Jun 02 21:41:07.345736 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:07.345679 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4g892" podStartSLOduration=3.382888325 podStartE2EDuration="4.345661173s" podCreationTimestamp="2026-06-02 21:41:03 +0000 UTC" firstStartedPulling="2026-06-02 21:41:04.483342667 +0000 UTC m=+92.284703620" lastFinishedPulling="2026-06-02 21:41:05.446115509 +0000 UTC m=+93.247476468" observedRunningTime="2026-06-02 21:41:07.344235885 +0000 UTC m=+95.145596860" watchObservedRunningTime="2026-06-02 21:41:07.345661173 +0000 UTC m=+95.147022147" Jun 02 21:41:09.224136 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:09.224107 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4q595" Jun 02 21:41:10.986448 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.986417 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n5lr4"] Jun 02 21:41:10.991143 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.991118 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:10.993298 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.993274 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 21:41:10.993474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.993451 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-l2892\"" Jun 02 21:41:10.994239 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.994220 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 21:41:10.994330 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.994283 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 21:41:10.994330 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:10.994322 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 21:41:11.003493 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.003471 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5lr4"] Jun 02 21:41:11.053168 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.053130 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15732505-5698-4bdf-8983-7c4f10a6394d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.053168 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.053170 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.053374 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.053201 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15732505-5698-4bdf-8983-7c4f10a6394d-crio-socket\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.053374 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.053276 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15732505-5698-4bdf-8983-7c4f10a6394d-data-volume\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.053374 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.053351 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq77t\" (UniqueName: \"kubernetes.io/projected/15732505-5698-4bdf-8983-7c4f10a6394d-kube-api-access-wq77t\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.096479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.096400 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:41:11.153895 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.153857 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wq77t\" (UniqueName: \"kubernetes.io/projected/15732505-5698-4bdf-8983-7c4f10a6394d-kube-api-access-wq77t\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154091 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.153911 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15732505-5698-4bdf-8983-7c4f10a6394d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154091 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.153931 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154091 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.153956 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15732505-5698-4bdf-8983-7c4f10a6394d-crio-socket\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154091 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.153984 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15732505-5698-4bdf-8983-7c4f10a6394d-data-volume\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154278 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.154090 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/15732505-5698-4bdf-8983-7c4f10a6394d-crio-socket\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154278 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:41:11.154104 2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:41:11.154278 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:41:11.154199 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls podName:15732505-5698-4bdf-8983-7c4f10a6394d nodeName:}" failed. No retries permitted until 2026-06-02 21:41:11.654175758 +0000 UTC m=+99.455536724 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n5lr4" (UID: "15732505-5698-4bdf-8983-7c4f10a6394d") : secret "insights-runtime-extractor-tls" not found Jun 02 21:41:11.154376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.154338 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/15732505-5698-4bdf-8983-7c4f10a6394d-data-volume\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.154471 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.154454 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/15732505-5698-4bdf-8983-7c4f10a6394d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.172701 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.172675 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq77t\" (UniqueName: \"kubernetes.io/projected/15732505-5698-4bdf-8983-7c4f10a6394d-kube-api-access-wq77t\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.658208 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.658170 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.660504 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.660482 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/15732505-5698-4bdf-8983-7c4f10a6394d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n5lr4\" (UID: \"15732505-5698-4bdf-8983-7c4f10a6394d\") " pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:11.900116 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:11.900080 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n5lr4" Jun 02 21:41:12.061798 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:12.059902 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n5lr4"] Jun 02 21:41:12.063141 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:41:12.063110 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15732505_5698_4bdf_8983_7c4f10a6394d.slice/crio-1bea87e2e44eeb1e06ba48f78793a84b5bc80b8ac438e60d7475a1e01700223e WatchSource:0}: Error finding container 1bea87e2e44eeb1e06ba48f78793a84b5bc80b8ac438e60d7475a1e01700223e: Status 404 returned error can't find the container with id 1bea87e2e44eeb1e06ba48f78793a84b5bc80b8ac438e60d7475a1e01700223e Jun 02 21:41:12.335514 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:12.335426 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5lr4" event={"ID":"15732505-5698-4bdf-8983-7c4f10a6394d","Type":"ContainerStarted","Data":"7d24d3c6e58ac8db3e4dc444248f17d09e5b5923d8b8df38391ecce1b6a3f8cc"} Jun 02 21:41:12.335514 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:12.335468 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5lr4" event={"ID":"15732505-5698-4bdf-8983-7c4f10a6394d","Type":"ContainerStarted","Data":"1bea87e2e44eeb1e06ba48f78793a84b5bc80b8ac438e60d7475a1e01700223e"} Jun 02 21:41:13.340361 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:13.340317 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5lr4" event={"ID":"15732505-5698-4bdf-8983-7c4f10a6394d","Type":"ContainerStarted","Data":"8364548f46a32b8087ec623e3a242034501d397193575e4c39d5c5eb6d05a0e2"} Jun 02 21:41:14.345557 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:14.345518 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n5lr4" event={"ID":"15732505-5698-4bdf-8983-7c4f10a6394d","Type":"ContainerStarted","Data":"11c17662ac9ec661e6300fa2a5ced8ce087fea94c41d9fe8172096d53814ce94"} Jun 02 21:41:14.368382 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:14.368332 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n5lr4" podStartSLOduration=2.250253779 podStartE2EDuration="4.368317086s" podCreationTimestamp="2026-06-02 21:41:10 +0000 UTC" firstStartedPulling="2026-06-02 21:41:12.120362012 +0000 UTC m=+99.921722972" lastFinishedPulling="2026-06-02 21:41:14.238425327 +0000 UTC m=+102.039786279" observedRunningTime="2026-06-02 21:41:14.367244347 +0000 UTC m=+102.168605335" watchObservedRunningTime="2026-06-02 21:41:14.368317086 +0000 UTC m=+102.169678061" Jun 02 21:41:32.400164 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:32.400131 2558 generic.go:358] "Generic (PLEG): container finished" podID="8796e9bb-846a-48ef-88be-45ac60ffc2fe" containerID="22491f55ebc5a69d5576de0addd5a7c154f07918595ebe19f772ac7d3015d821" exitCode=0 Jun 02 21:41:32.400593 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:32.400203 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" event={"ID":"8796e9bb-846a-48ef-88be-45ac60ffc2fe","Type":"ContainerDied","Data":"22491f55ebc5a69d5576de0addd5a7c154f07918595ebe19f772ac7d3015d821"} Jun 02 21:41:32.400593 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:32.400544 2558 scope.go:117] "RemoveContainer" containerID="22491f55ebc5a69d5576de0addd5a7c154f07918595ebe19f772ac7d3015d821" Jun 02 21:41:33.404230 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:33.404193 2558 generic.go:358] "Generic (PLEG): container finished" podID="628721dd-9b4c-4673-9540-7457f195b066" containerID="bc85590def886e1d1a9f03910d3b064b3ead14d87cc32e901b08c2995dbb6c0f" exitCode=0 Jun 02 21:41:33.404656 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:33.404268 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" event={"ID":"628721dd-9b4c-4673-9540-7457f195b066","Type":"ContainerDied","Data":"bc85590def886e1d1a9f03910d3b064b3ead14d87cc32e901b08c2995dbb6c0f"} Jun 02 21:41:33.404656 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:33.404605 2558 scope.go:117] "RemoveContainer" containerID="bc85590def886e1d1a9f03910d3b064b3ead14d87cc32e901b08c2995dbb6c0f" Jun 02 21:41:33.406065 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:33.406039 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-96ddc787c-2swqs" event={"ID":"8796e9bb-846a-48ef-88be-45ac60ffc2fe","Type":"ContainerStarted","Data":"148fcf1baab36af5cab7a73a0eca5446c11149020f0854bfe58722d0ef9602d6"} Jun 02 21:41:34.410627 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:34.410594 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-69877444c-2n4nc" event={"ID":"628721dd-9b4c-4673-9540-7457f195b066","Type":"ContainerStarted","Data":"256f40bb5a1defb4f9b7165353052b31b9aad44958937536a4d8170e8a01a5b2"} Jun 02 21:41:36.116844 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.116800 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerName="registry" containerID="cri-o://2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58" gracePeriod=30 Jun 02 21:41:36.361383 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.361355 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:41:36.417822 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.417795 2558 generic.go:358] "Generic (PLEG): container finished" podID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerID="2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58" exitCode=0 Jun 02 21:41:36.418017 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.417854 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" Jun 02 21:41:36.418017 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.417888 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" event={"ID":"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46","Type":"ContainerDied","Data":"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58"} Jun 02 21:41:36.418017 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.417931 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c9c4d9d-r7d8x" event={"ID":"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46","Type":"ContainerDied","Data":"b814129808b10a0e543ff1935b9765380507ddfca32450efd5cc07b6e25dee46"} Jun 02 21:41:36.418017 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.417948 2558 scope.go:117] "RemoveContainer" containerID="2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58" Jun 02 21:41:36.426138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.426117 2558 scope.go:117] "RemoveContainer" containerID="2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58" Jun 02 21:41:36.426430 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:41:36.426407 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58\": container with ID starting with 2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58 not found: ID does not exist" containerID="2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58" Jun 02 21:41:36.426496 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.426439 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58"} err="failed to get container status \"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58\": rpc error: code = NotFound desc = could not find container \"2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58\": container with ID starting with 2419637bd55a1232c1c3e7096138b3d6e1849a3618729e96f68b0f77f0aa1c58 not found: ID does not exist" Jun 02 21:41:36.465764 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.465724 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.465914 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.465791 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.465914 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.465811 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466022 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.465907 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4ljz\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466022 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.465954 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466135 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.466047 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466135 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.466070 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466135 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.466108 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted\") pod \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\" (UID: \"17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46\") " Jun 02 21:41:36.466316 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.466289 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:41:36.466746 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.466720 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:41:36.468846 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.468817 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:41:36.468960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.468881 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:41:36.469045 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.468956 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz" (OuterVolumeSpecName: "kube-api-access-j4ljz") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "kube-api-access-j4ljz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:41:36.469045 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.468961 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:41:36.469045 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.469017 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:41:36.474877 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.474854 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" (UID: "17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:41:36.567662 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567618 2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-bound-sa-token\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567662 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567655 2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-certificates\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567672 2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-ca-trust-extracted\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567687 2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-image-registry-private-configuration\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567700 2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-installation-pull-secrets\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567713 2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-trusted-ca\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567727 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4ljz\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-kube-api-access-j4ljz\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.567869 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.567741 2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46-registry-tls\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:41:36.742766 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.742731 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:41:36.748655 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.748629 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-58c9c4d9d-r7d8x"] Jun 02 21:41:36.811625 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:36.811590 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" path="/var/lib/kubelet/pods/17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46/volumes" Jun 02 21:41:41.615161 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:41.615117 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:41:41.617410 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:41.617385 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afdad777-7975-4eef-bc20-ef9878174b53-metrics-certs\") pod \"network-metrics-daemon-s6gxd\" (UID: \"afdad777-7975-4eef-bc20-ef9878174b53\") " pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:41:41.635455 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:41.635425 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8znwf\"" Jun 02 21:41:41.643541 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:41.643522 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s6gxd" Jun 02 21:41:41.774688 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:41.774654 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s6gxd"] Jun 02 21:41:41.777903 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:41:41.777871 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafdad777_7975_4eef_bc20_ef9878174b53.slice/crio-bbad6c9941bfe832f4a73fbb6e09214c1aa03280d74bcc8ca1249406fb30fe62 WatchSource:0}: Error finding container bbad6c9941bfe832f4a73fbb6e09214c1aa03280d74bcc8ca1249406fb30fe62: Status 404 returned error can't find the container with id bbad6c9941bfe832f4a73fbb6e09214c1aa03280d74bcc8ca1249406fb30fe62 Jun 02 21:41:42.442442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:42.442398 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s6gxd" event={"ID":"afdad777-7975-4eef-bc20-ef9878174b53","Type":"ContainerStarted","Data":"bbad6c9941bfe832f4a73fbb6e09214c1aa03280d74bcc8ca1249406fb30fe62"} Jun 02 21:41:43.447086 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:43.447051 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s6gxd" event={"ID":"afdad777-7975-4eef-bc20-ef9878174b53","Type":"ContainerStarted","Data":"83c444b002e293cb1dedddea17aebb55df6a6eaeb088cf504f39af5b0bde03c3"} Jun 02 21:41:43.447086 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:43.447089 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s6gxd" event={"ID":"afdad777-7975-4eef-bc20-ef9878174b53","Type":"ContainerStarted","Data":"24270a22b037bb492c34733727b751c1adc898a0e62b0ff6332030a9dbf5da4e"} Jun 02 21:41:43.465731 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:43.465674 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s6gxd" podStartSLOduration=130.243599182 podStartE2EDuration="2m11.465659907s" podCreationTimestamp="2026-06-02 21:39:32 +0000 UTC" firstStartedPulling="2026-06-02 21:41:41.780123163 +0000 UTC m=+129.581484116" lastFinishedPulling="2026-06-02 21:41:43.002183885 +0000 UTC m=+130.803544841" observedRunningTime="2026-06-02 21:41:43.464227577 +0000 UTC m=+131.265588552" watchObservedRunningTime="2026-06-02 21:41:43.465659907 +0000 UTC m=+131.267020881" Jun 02 21:41:56.576802 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:41:56.576761 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" podUID="adf7837a-7e9f-4612-a770-52b31b7a06d9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:42:06.576373 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:06.576330 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" podUID="adf7837a-7e9f-4612-a770-52b31b7a06d9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:42:16.576976 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:16.576934 2558 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" podUID="adf7837a-7e9f-4612-a770-52b31b7a06d9" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:42:16.577467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:16.577037 2558 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" Jun 02 21:42:16.577533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:16.577515 2558 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"14078beffe5b28b6fdfb5cda0d5ad93270a63f115ace5933d0af444534b5f65e"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 02 21:42:16.577572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:16.577555 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" podUID="adf7837a-7e9f-4612-a770-52b31b7a06d9" containerName="service-proxy" containerID="cri-o://14078beffe5b28b6fdfb5cda0d5ad93270a63f115ace5933d0af444534b5f65e" gracePeriod=30 Jun 02 21:42:17.544302 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:17.544257 2558 generic.go:358] "Generic (PLEG): container finished" podID="adf7837a-7e9f-4612-a770-52b31b7a06d9" containerID="14078beffe5b28b6fdfb5cda0d5ad93270a63f115ace5933d0af444534b5f65e" exitCode=2 Jun 02 21:42:17.544474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:17.544342 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerDied","Data":"14078beffe5b28b6fdfb5cda0d5ad93270a63f115ace5933d0af444534b5f65e"} Jun 02 21:42:17.544474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:42:17.544393 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7fcf49b977-v56q9" event={"ID":"adf7837a-7e9f-4612-a770-52b31b7a06d9","Type":"ContainerStarted","Data":"d6665f2cbb48260c2e9c5110beef445e373b5d6f1b721de3ec6908d9b4978008"} Jun 02 21:44:32.713624 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:32.713595 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:44:32.714159 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:32.713936 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:44:32.722199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:32.722171 2558 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 21:44:55.632243 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.632199 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g"] Jun 02 21:44:55.634731 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.632485 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerName="registry" Jun 02 21:44:55.634731 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.632502 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerName="registry" Jun 02 21:44:55.634731 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.632573 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="17784d2f-8b5f-4eb9-a21d-1b59a8ca3e46" containerName="registry" Jun 02 21:44:55.635729 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.635708 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.638101 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.638075 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 21:44:55.638205 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.638075 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 21:44:55.638711 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.638689 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 21:44:55.638782 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.638728 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-nvhhq\"" Jun 02 21:44:55.639590 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.639575 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 21:44:55.655668 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.655638 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g"] Jun 02 21:44:55.695303 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.695263 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cwmr\" (UniqueName: \"kubernetes.io/projected/61f01710-6877-4245-ad32-b5b4565054c1-kube-api-access-7cwmr\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.695467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.695310 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-manifests\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.695467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.695329 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.695467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.695419 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-tmp\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.695467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.695451 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796178 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796136 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7cwmr\" (UniqueName: \"kubernetes.io/projected/61f01710-6877-4245-ad32-b5b4565054c1-kube-api-access-7cwmr\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796353 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796187 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-manifests\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796353 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796216 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796353 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796266 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-tmp\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796353 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796296 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.796660 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.796636 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-manifests\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.798659 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.798625 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/61f01710-6877-4245-ad32-b5b4565054c1-tmp\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.798839 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.798823 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.798897 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.798880 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/61f01710-6877-4245-ad32-b5b4565054c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.805213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.805189 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cwmr\" (UniqueName: \"kubernetes.io/projected/61f01710-6877-4245-ad32-b5b4565054c1-kube-api-access-7cwmr\") pod \"opendatahub-operator-controller-manager-6d44dd895c-ntq4g\" (UID: \"61f01710-6877-4245-ad32-b5b4565054c1\") " pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:55.945540 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:55.945492 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:44:56.099529 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:56.099492 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g"] Jun 02 21:44:56.102721 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:44:56.102689 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f01710_6877_4245_ad32_b5b4565054c1.slice/crio-013a23662b3f6a503102af761c8e7a2b05e0c96a2645cb3d82d48c8d43c2da8f WatchSource:0}: Error finding container 013a23662b3f6a503102af761c8e7a2b05e0c96a2645cb3d82d48c8d43c2da8f: Status 404 returned error can't find the container with id 013a23662b3f6a503102af761c8e7a2b05e0c96a2645cb3d82d48c8d43c2da8f Jun 02 21:44:56.104416 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:56.104398 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:44:56.987795 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:56.987735 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" event={"ID":"61f01710-6877-4245-ad32-b5b4565054c1","Type":"ContainerStarted","Data":"013a23662b3f6a503102af761c8e7a2b05e0c96a2645cb3d82d48c8d43c2da8f"} Jun 02 21:44:58.994678 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:58.994636 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" event={"ID":"61f01710-6877-4245-ad32-b5b4565054c1","Type":"ContainerStarted","Data":"eb3ad2c5de0ac9134f0b434cbcd44c3b47826abaa3a151b8d51906ced559e6bf"} Jun 02 21:44:59.999224 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:59.999190 2558 generic.go:358] "Generic (PLEG): container finished" podID="61f01710-6877-4245-ad32-b5b4565054c1" containerID="eb3ad2c5de0ac9134f0b434cbcd44c3b47826abaa3a151b8d51906ced559e6bf" exitCode=0 Jun 02 21:44:59.999702 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:44:59.999241 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" event={"ID":"61f01710-6877-4245-ad32-b5b4565054c1","Type":"ContainerDied","Data":"eb3ad2c5de0ac9134f0b434cbcd44c3b47826abaa3a151b8d51906ced559e6bf"} Jun 02 21:45:01.004295 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:01.004256 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" event={"ID":"61f01710-6877-4245-ad32-b5b4565054c1","Type":"ContainerStarted","Data":"65d56804f3497a53d0186f319e34e3d8638b00f8ee91746026fd3313e7bd05e3"} Jun 02 21:45:01.004657 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:01.004393 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:45:01.025658 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:01.025580 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" podStartSLOduration=1.822838567 podStartE2EDuration="6.025566242s" podCreationTimestamp="2026-06-02 21:44:55 +0000 UTC" firstStartedPulling="2026-06-02 21:44:56.104520214 +0000 UTC m=+323.905881168" lastFinishedPulling="2026-06-02 21:45:00.307247875 +0000 UTC m=+328.108608843" observedRunningTime="2026-06-02 21:45:01.024545523 +0000 UTC m=+328.825906499" watchObservedRunningTime="2026-06-02 21:45:01.025566242 +0000 UTC m=+328.826927213" Jun 02 21:45:12.009391 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:12.009354 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d44dd895c-ntq4g" Jun 02 21:45:15.543543 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.543509 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p"] Jun 02 21:45:15.557521 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.557486 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.561775 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.561741 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 21:45:15.561917 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.561756 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 21:45:15.562300 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.562283 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 21:45:15.562344 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.562283 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 21:45:15.574931 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.574896 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p"] Jun 02 21:45:15.665606 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.665571 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bad53864-7d52-4c10-9f47-26642ed53d69-tmp\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.665805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.665637 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqjfx\" (UniqueName: \"kubernetes.io/projected/bad53864-7d52-4c10-9f47-26642ed53d69-kube-api-access-kqjfx\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.665805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.665693 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.762343 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.762302 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-d76t4"] Jun 02 21:45:15.765847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.765822 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:15.766309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.766285 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bad53864-7d52-4c10-9f47-26642ed53d69-tmp\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.766414 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.766339 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kqjfx\" (UniqueName: \"kubernetes.io/projected/bad53864-7d52-4c10-9f47-26642ed53d69-kube-api-access-kqjfx\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.766414 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.766376 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.766512 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:15.766473 2558 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 02 21:45:15.766567 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:15.766530 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs podName:bad53864-7d52-4c10-9f47-26642ed53d69 nodeName:}" failed. No retries permitted until 2026-06-02 21:45:16.266515069 +0000 UTC m=+344.067876022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs") pod "kube-auth-proxy-567f8cc9bf-xqt6p" (UID: "bad53864-7d52-4c10-9f47-26642ed53d69") : secret "kube-auth-proxy-tls" not found Jun 02 21:45:15.768602 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.768577 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bad53864-7d52-4c10-9f47-26642ed53d69-tmp\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.771053 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.771031 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 02 21:45:15.771166 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.771060 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-knll8\"" Jun 02 21:45:15.786378 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.786351 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-d76t4"] Jun 02 21:45:15.796637 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.796560 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqjfx\" (UniqueName: \"kubernetes.io/projected/bad53864-7d52-4c10-9f47-26642ed53d69-kube-api-access-kqjfx\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:15.867025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.866973 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltfq\" (UniqueName: \"kubernetes.io/projected/a243b531-9ec9-4e51-93f7-45ecc70e24b8-kube-api-access-9ltfq\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:15.867194 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.867127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:15.967935 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.967884 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:15.968134 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.967967 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltfq\" (UniqueName: \"kubernetes.io/projected/a243b531-9ec9-4e51-93f7-45ecc70e24b8-kube-api-access-9ltfq\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:15.968134 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:15.968086 2558 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 02 21:45:15.968259 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:15.968177 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert podName:a243b531-9ec9-4e51-93f7-45ecc70e24b8 nodeName:}" failed. No retries permitted until 2026-06-02 21:45:16.46815597 +0000 UTC m=+344.269516927 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert") pod "odh-model-controller-76588b95-d76t4" (UID: "a243b531-9ec9-4e51-93f7-45ecc70e24b8") : secret "odh-model-controller-webhook-cert" not found Jun 02 21:45:15.985060 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:15.985025 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltfq\" (UniqueName: \"kubernetes.io/projected/a243b531-9ec9-4e51-93f7-45ecc70e24b8-kube-api-access-9ltfq\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:16.271231 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.271186 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:16.273730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.273699 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad53864-7d52-4c10-9f47-26642ed53d69-tls-certs\") pod \"kube-auth-proxy-567f8cc9bf-xqt6p\" (UID: \"bad53864-7d52-4c10-9f47-26642ed53d69\") " pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:16.467161 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.467124 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" Jun 02 21:45:16.473240 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.473209 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:16.476065 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.476042 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a243b531-9ec9-4e51-93f7-45ecc70e24b8-cert\") pod \"odh-model-controller-76588b95-d76t4\" (UID: \"a243b531-9ec9-4e51-93f7-45ecc70e24b8\") " pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:16.605178 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.605151 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p"] Jun 02 21:45:16.608101 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:16.608069 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad53864_7d52_4c10_9f47_26642ed53d69.slice/crio-f7d22f8c9f7520f62822b5e6675e80158def96092f4514c82034d64988f47aae WatchSource:0}: Error finding container f7d22f8c9f7520f62822b5e6675e80158def96092f4514c82034d64988f47aae: Status 404 returned error can't find the container with id f7d22f8c9f7520f62822b5e6675e80158def96092f4514c82034d64988f47aae Jun 02 21:45:16.689895 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.689861 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:16.825237 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:16.825211 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-d76t4"] Jun 02 21:45:16.827637 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:16.827606 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda243b531_9ec9_4e51_93f7_45ecc70e24b8.slice/crio-32480c09b2ccdd50791ec9786139c86a7cf57e261b8fe6661d432ae6b8f6353e WatchSource:0}: Error finding container 32480c09b2ccdd50791ec9786139c86a7cf57e261b8fe6661d432ae6b8f6353e: Status 404 returned error can't find the container with id 32480c09b2ccdd50791ec9786139c86a7cf57e261b8fe6661d432ae6b8f6353e Jun 02 21:45:17.051195 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:17.051156 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-d76t4" event={"ID":"a243b531-9ec9-4e51-93f7-45ecc70e24b8","Type":"ContainerStarted","Data":"32480c09b2ccdd50791ec9786139c86a7cf57e261b8fe6661d432ae6b8f6353e"} Jun 02 21:45:17.052199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:17.052170 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" event={"ID":"bad53864-7d52-4c10-9f47-26642ed53d69","Type":"ContainerStarted","Data":"f7d22f8c9f7520f62822b5e6675e80158def96092f4514c82034d64988f47aae"} Jun 02 21:45:22.075585 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:22.075480 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" event={"ID":"bad53864-7d52-4c10-9f47-26642ed53d69","Type":"ContainerStarted","Data":"1bc362c90913cb0cd522a8af71815ecf1d7051b8832ad0eb78d473ea256e9f05"} Jun 02 21:45:22.077023 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:22.076970 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-d76t4" event={"ID":"a243b531-9ec9-4e51-93f7-45ecc70e24b8","Type":"ContainerStarted","Data":"416e5cf5402b5f26d43e362635dadfba64bc95b487ab0b103cb98eb2241e5ec0"} Jun 02 21:45:22.077127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:22.077100 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:22.157571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:22.157338 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-567f8cc9bf-xqt6p" podStartSLOduration=2.799154222 podStartE2EDuration="7.157322191s" podCreationTimestamp="2026-06-02 21:45:15 +0000 UTC" firstStartedPulling="2026-06-02 21:45:16.60961336 +0000 UTC m=+344.410974314" lastFinishedPulling="2026-06-02 21:45:20.96778132 +0000 UTC m=+348.769142283" observedRunningTime="2026-06-02 21:45:22.122219138 +0000 UTC m=+349.923580113" watchObservedRunningTime="2026-06-02 21:45:22.157322191 +0000 UTC m=+349.958683166" Jun 02 21:45:22.158306 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:22.158266 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-d76t4" podStartSLOduration=2.984463778 podStartE2EDuration="7.158256632s" podCreationTimestamp="2026-06-02 21:45:15 +0000 UTC" firstStartedPulling="2026-06-02 21:45:16.828904261 +0000 UTC m=+344.630265215" lastFinishedPulling="2026-06-02 21:45:21.002697104 +0000 UTC m=+348.804058069" observedRunningTime="2026-06-02 21:45:22.156902782 +0000 UTC m=+349.958263761" watchObservedRunningTime="2026-06-02 21:45:22.158256632 +0000 UTC m=+349.959617606" Jun 02 21:45:27.331587 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.331547 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr"] Jun 02 21:45:27.334501 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.334484 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.338634 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.338604 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-229ns\"" Jun 02 21:45:27.338841 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.338821 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 02 21:45:27.354608 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.354568 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2"] Jun 02 21:45:27.357730 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.357709 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr"] Jun 02 21:45:27.357843 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.357823 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.360713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.360666 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-t6hw2\"" Jun 02 21:45:27.360919 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.360904 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 02 21:45:27.373282 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.373251 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.373418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.373301 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mv8f\" (UniqueName: \"kubernetes.io/projected/8a49494c-6ab7-4b0c-bccf-0a9def83c472-kube-api-access-6mv8f\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.373418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.373362 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27lh6\" (UniqueName: \"kubernetes.io/projected/29d90541-7c1c-4b27-9f5e-17b028ef3f43-kube-api-access-27lh6\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.373418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.373406 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.387161 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.387123 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2"] Jun 02 21:45:27.474699 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.474665 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.474896 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.474730 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.474896 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.474767 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6mv8f\" (UniqueName: \"kubernetes.io/projected/8a49494c-6ab7-4b0c-bccf-0a9def83c472-kube-api-access-6mv8f\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.474896 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.474792 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-27lh6\" (UniqueName: \"kubernetes.io/projected/29d90541-7c1c-4b27-9f5e-17b028ef3f43-kube-api-access-27lh6\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.474896 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:27.474827 2558 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 02 21:45:27.474896 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:27.474898 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert podName:8a49494c-6ab7-4b0c-bccf-0a9def83c472 nodeName:}" failed. No retries permitted until 2026-06-02 21:45:27.974881079 +0000 UTC m=+355.776242031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert") pod "kserve-controller-manager-7c99ffcdcd-h5hnr" (UID: "8a49494c-6ab7-4b0c-bccf-0a9def83c472") : secret "kserve-webhook-server-cert" not found Jun 02 21:45:27.475131 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:27.474902 2558 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 02 21:45:27.475131 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:45:27.474963 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert podName:29d90541-7c1c-4b27-9f5e-17b028ef3f43 nodeName:}" failed. No retries permitted until 2026-06-02 21:45:27.974946192 +0000 UTC m=+355.776307150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert") pod "llmisvc-controller-manager-c6b9847d7-tw9l2" (UID: "29d90541-7c1c-4b27-9f5e-17b028ef3f43") : secret "llmisvc-webhook-server-cert" not found Jun 02 21:45:27.489582 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.489554 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27lh6\" (UniqueName: \"kubernetes.io/projected/29d90541-7c1c-4b27-9f5e-17b028ef3f43-kube-api-access-27lh6\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.490045 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.490022 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mv8f\" (UniqueName: \"kubernetes.io/projected/8a49494c-6ab7-4b0c-bccf-0a9def83c472-kube-api-access-6mv8f\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.977723 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.977673 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.977912 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.977743 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:27.980257 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.980224 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/29d90541-7c1c-4b27-9f5e-17b028ef3f43-cert\") pod \"llmisvc-controller-manager-c6b9847d7-tw9l2\" (UID: \"29d90541-7c1c-4b27-9f5e-17b028ef3f43\") " pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:27.980257 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:27.980249 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a49494c-6ab7-4b0c-bccf-0a9def83c472-cert\") pod \"kserve-controller-manager-7c99ffcdcd-h5hnr\" (UID: \"8a49494c-6ab7-4b0c-bccf-0a9def83c472\") " pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:28.245648 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:28.245547 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:28.269220 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:28.269188 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:28.391890 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:28.391825 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr"] Jun 02 21:45:28.394265 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:28.394226 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a49494c_6ab7_4b0c_bccf_0a9def83c472.slice/crio-5ccd26c25c94c5a189242f694f8575c6660033021463776830815bf85f9556bc WatchSource:0}: Error finding container 5ccd26c25c94c5a189242f694f8575c6660033021463776830815bf85f9556bc: Status 404 returned error can't find the container with id 5ccd26c25c94c5a189242f694f8575c6660033021463776830815bf85f9556bc Jun 02 21:45:28.419030 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:28.419000 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2"] Jun 02 21:45:28.420930 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:28.420900 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod29d90541_7c1c_4b27_9f5e_17b028ef3f43.slice/crio-f4365bf80b37ad3db113d7887f0d94d81e72216b476716cadb1df53e30eed597 WatchSource:0}: Error finding container f4365bf80b37ad3db113d7887f0d94d81e72216b476716cadb1df53e30eed597: Status 404 returned error can't find the container with id f4365bf80b37ad3db113d7887f0d94d81e72216b476716cadb1df53e30eed597 Jun 02 21:45:29.105141 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:29.105101 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" event={"ID":"8a49494c-6ab7-4b0c-bccf-0a9def83c472","Type":"ContainerStarted","Data":"5ccd26c25c94c5a189242f694f8575c6660033021463776830815bf85f9556bc"} Jun 02 21:45:29.106142 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:29.106113 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" event={"ID":"29d90541-7c1c-4b27-9f5e-17b028ef3f43","Type":"ContainerStarted","Data":"f4365bf80b37ad3db113d7887f0d94d81e72216b476716cadb1df53e30eed597"} Jun 02 21:45:31.724027 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.723983 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-8c7bb9f47-zmrwq"] Jun 02 21:45:31.728627 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.728603 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.746567 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.746543 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 02 21:45:31.746712 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.746695 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fqfwq\"" Jun 02 21:45:31.776364 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.776331 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-8c7bb9f47-zmrwq"] Jun 02 21:45:31.814895 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.814856 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/43794281-b96c-4866-acbb-e7af09d6c096-redirect-config\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.815100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.814962 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99vhk\" (UniqueName: \"kubernetes.io/projected/43794281-b96c-4866-acbb-e7af09d6c096-kube-api-access-99vhk\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.916065 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.916020 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/43794281-b96c-4866-acbb-e7af09d6c096-redirect-config\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.916399 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.916123 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-99vhk\" (UniqueName: \"kubernetes.io/projected/43794281-b96c-4866-acbb-e7af09d6c096-kube-api-access-99vhk\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.916763 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.916737 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/43794281-b96c-4866-acbb-e7af09d6c096-redirect-config\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:31.999625 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:31.999551 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99vhk\" (UniqueName: \"kubernetes.io/projected/43794281-b96c-4866-acbb-e7af09d6c096-kube-api-access-99vhk\") pod \"dashboard-redirect-8c7bb9f47-zmrwq\" (UID: \"43794281-b96c-4866-acbb-e7af09d6c096\") " pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:32.054148 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.054111 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" Jun 02 21:45:32.121839 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.121784 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" event={"ID":"29d90541-7c1c-4b27-9f5e-17b028ef3f43","Type":"ContainerStarted","Data":"550cceaad6e02a50c431655a30c347241d61dbf79dc878c878d056a1a7c85530"} Jun 02 21:45:32.122040 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.121951 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:45:32.123453 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.123417 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" event={"ID":"8a49494c-6ab7-4b0c-bccf-0a9def83c472","Type":"ContainerStarted","Data":"2b890f201a08c17720aa321ce694c9c67eb586914cc22a8d90a6ef556177dfcf"} Jun 02 21:45:32.123589 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.123569 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:45:32.165638 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.165577 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" podStartSLOduration=1.894144684 podStartE2EDuration="5.165561205s" podCreationTimestamp="2026-06-02 21:45:27 +0000 UTC" firstStartedPulling="2026-06-02 21:45:28.422276507 +0000 UTC m=+356.223637459" lastFinishedPulling="2026-06-02 21:45:31.693693023 +0000 UTC m=+359.495053980" observedRunningTime="2026-06-02 21:45:32.163984264 +0000 UTC m=+359.965345255" watchObservedRunningTime="2026-06-02 21:45:32.165561205 +0000 UTC m=+359.966922229" Jun 02 21:45:32.215803 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.215738 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" podStartSLOduration=1.920990196 podStartE2EDuration="5.215720741s" podCreationTimestamp="2026-06-02 21:45:27 +0000 UTC" firstStartedPulling="2026-06-02 21:45:28.395640219 +0000 UTC m=+356.197001171" lastFinishedPulling="2026-06-02 21:45:31.690370744 +0000 UTC m=+359.491731716" observedRunningTime="2026-06-02 21:45:32.209600365 +0000 UTC m=+360.010961343" watchObservedRunningTime="2026-06-02 21:45:32.215720741 +0000 UTC m=+360.017081716" Jun 02 21:45:32.241582 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:32.241544 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-8c7bb9f47-zmrwq"] Jun 02 21:45:32.247156 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:32.247124 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43794281_b96c_4866_acbb_e7af09d6c096.slice/crio-a1f0fd4b49593183f58f2a03238976fa32365f497139300ed918418827fdff54 WatchSource:0}: Error finding container a1f0fd4b49593183f58f2a03238976fa32365f497139300ed918418827fdff54: Status 404 returned error can't find the container with id a1f0fd4b49593183f58f2a03238976fa32365f497139300ed918418827fdff54 Jun 02 21:45:33.082816 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:33.082785 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-d76t4" Jun 02 21:45:33.129036 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:33.128978 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" event={"ID":"43794281-b96c-4866-acbb-e7af09d6c096","Type":"ContainerStarted","Data":"a1f0fd4b49593183f58f2a03238976fa32365f497139300ed918418827fdff54"} Jun 02 21:45:34.103808 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.103769 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-npb29"] Jun 02 21:45:34.107225 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.107199 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.112314 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.112285 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-dtqz2\"" Jun 02 21:45:34.112537 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.112520 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 02 21:45:34.112804 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.112787 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 02 21:45:34.120982 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.120942 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-npb29"] Jun 02 21:45:34.147182 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.142634 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/0194f086-b458-4767-ba0a-d77613e79b1e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.147182 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.142710 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd4nk\" (UniqueName: \"kubernetes.io/projected/0194f086-b458-4767-ba0a-d77613e79b1e-kube-api-access-qd4nk\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.243148 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.243115 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/0194f086-b458-4767-ba0a-d77613e79b1e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.243358 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.243201 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qd4nk\" (UniqueName: \"kubernetes.io/projected/0194f086-b458-4767-ba0a-d77613e79b1e-kube-api-access-qd4nk\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.245847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.245823 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/0194f086-b458-4767-ba0a-d77613e79b1e-operator-config\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.271127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.271088 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd4nk\" (UniqueName: \"kubernetes.io/projected/0194f086-b458-4767-ba0a-d77613e79b1e-kube-api-access-qd4nk\") pod \"servicemesh-operator3-55f49c5f94-npb29\" (UID: \"0194f086-b458-4767-ba0a-d77613e79b1e\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.420281 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.420233 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:34.602481 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:34.602444 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-npb29"] Jun 02 21:45:34.605729 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:34.605698 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0194f086_b458_4767_ba0a_d77613e79b1e.slice/crio-a58870e65d14719bb4e044d56012cae4d36887a1a796b3a64e6db156263de5d2 WatchSource:0}: Error finding container a58870e65d14719bb4e044d56012cae4d36887a1a796b3a64e6db156263de5d2: Status 404 returned error can't find the container with id a58870e65d14719bb4e044d56012cae4d36887a1a796b3a64e6db156263de5d2 Jun 02 21:45:35.162919 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:35.162880 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" event={"ID":"43794281-b96c-4866-acbb-e7af09d6c096","Type":"ContainerStarted","Data":"8d9544872133b0f5c329ca86944e6a4aa21b8970cc2a9ccb2570a18d0988e3c9"} Jun 02 21:45:35.170082 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:35.170037 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" event={"ID":"0194f086-b458-4767-ba0a-d77613e79b1e","Type":"ContainerStarted","Data":"a58870e65d14719bb4e044d56012cae4d36887a1a796b3a64e6db156263de5d2"} Jun 02 21:45:42.832216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:42.832162 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-8c7bb9f47-zmrwq" podStartSLOduration=9.966087179 podStartE2EDuration="11.832147592s" podCreationTimestamp="2026-06-02 21:45:31 +0000 UTC" firstStartedPulling="2026-06-02 21:45:32.248668804 +0000 UTC m=+360.050029757" lastFinishedPulling="2026-06-02 21:45:34.114729214 +0000 UTC m=+361.916090170" observedRunningTime="2026-06-02 21:45:35.189640806 +0000 UTC m=+362.991001781" watchObservedRunningTime="2026-06-02 21:45:42.832147592 +0000 UTC m=+370.633508574" Jun 02 21:45:48.213202 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.213159 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" event={"ID":"0194f086-b458-4767-ba0a-d77613e79b1e","Type":"ContainerStarted","Data":"079b92edf2c2fe0d4dc85cb164c2bbe8cec580f770484f79a4d2ace5071522e2"} Jun 02 21:45:48.213678 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.213276 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:45:48.236454 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.236389 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" podStartSLOduration=1.216316804 podStartE2EDuration="14.236370029s" podCreationTimestamp="2026-06-02 21:45:34 +0000 UTC" firstStartedPulling="2026-06-02 21:45:34.608094724 +0000 UTC m=+362.409455677" lastFinishedPulling="2026-06-02 21:45:47.628147949 +0000 UTC m=+375.429508902" observedRunningTime="2026-06-02 21:45:48.234144467 +0000 UTC m=+376.035505441" watchObservedRunningTime="2026-06-02 21:45:48.236370029 +0000 UTC m=+376.037731005" Jun 02 21:45:48.628912 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.628833 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts"] Jun 02 21:45:48.632166 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.632150 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.635237 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.635219 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 02 21:45:48.635413 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.635399 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 02 21:45:48.635474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.635415 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 02 21:45:48.635517 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.635401 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 02 21:45:48.635517 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.635487 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-x2lmv\"" Jun 02 21:45:48.648675 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.648648 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts"] Jun 02 21:45:48.773426 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm98b\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-kube-api-access-tm98b\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773433 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773523 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773554 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-cacerts\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773582 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-token\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773742 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773621 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9b891ec-69f2-4a6c-a1d0-68213d829018-local-certs\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.773742 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.773648 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874361 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874313 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874539 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874394 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tm98b\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-kube-api-access-tm98b\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874539 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874432 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874539 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874480 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874539 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874512 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-cacerts\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874753 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874546 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-token\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.874753 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.874581 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9b891ec-69f2-4a6c-a1d0-68213d829018-local-certs\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.875132 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.875102 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.876874 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.876839 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.877021 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.876929 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-cacerts\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.877286 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.877263 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.877372 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.877307 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9b891ec-69f2-4a6c-a1d0-68213d829018-local-certs\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.887412 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.887354 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm98b\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-kube-api-access-tm98b\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.888596 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.888570 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9b891ec-69f2-4a6c-a1d0-68213d829018-istio-token\") pod \"istiod-openshift-gateway-798958bb55-g88ts\" (UID: \"d9b891ec-69f2-4a6c-a1d0-68213d829018\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:48.942061 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:48.942018 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:49.098908 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:49.098876 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts"] Jun 02 21:45:49.102713 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:45:49.102685 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9b891ec_69f2_4a6c_a1d0_68213d829018.slice/crio-f3b0f5ac41ecbc48d5c87127467f50ddf906d93dfd0e1135189677d9850954cd WatchSource:0}: Error finding container f3b0f5ac41ecbc48d5c87127467f50ddf906d93dfd0e1135189677d9850954cd: Status 404 returned error can't find the container with id f3b0f5ac41ecbc48d5c87127467f50ddf906d93dfd0e1135189677d9850954cd Jun 02 21:45:49.218386 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:49.218341 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" event={"ID":"d9b891ec-69f2-4a6c-a1d0-68213d829018","Type":"ContainerStarted","Data":"f3b0f5ac41ecbc48d5c87127467f50ddf906d93dfd0e1135189677d9850954cd"} Jun 02 21:45:55.184380 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:55.184338 2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 02 21:45:55.184654 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:55.184425 2558 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 02 21:45:56.247337 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:56.247302 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" event={"ID":"d9b891ec-69f2-4a6c-a1d0-68213d829018","Type":"ContainerStarted","Data":"77c5e3ff61345a8f7aad91766a6e21a5324fba036539fd02d8907648473e8d8f"} Jun 02 21:45:56.247798 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:56.247501 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:56.249014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:56.248964 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" Jun 02 21:45:56.406064 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:56.406002 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-g88ts" podStartSLOduration=2.326725435 podStartE2EDuration="8.405968983s" podCreationTimestamp="2026-06-02 21:45:48 +0000 UTC" firstStartedPulling="2026-06-02 21:45:49.104848099 +0000 UTC m=+376.906209052" lastFinishedPulling="2026-06-02 21:45:55.184091647 +0000 UTC m=+382.985452600" observedRunningTime="2026-06-02 21:45:56.394828784 +0000 UTC m=+384.196189767" watchObservedRunningTime="2026-06-02 21:45:56.405968983 +0000 UTC m=+384.207329958" Jun 02 21:45:59.221220 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:45:59.221184 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-npb29" Jun 02 21:46:03.132192 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:03.132158 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-c6b9847d7-tw9l2" Jun 02 21:46:03.135536 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:03.135506 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-7c99ffcdcd-h5hnr" Jun 02 21:46:52.324811 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.324726 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt"] Jun 02 21:46:52.328140 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.328121 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:52.330977 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.330924 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 21:46:52.331139 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.331058 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 21:46:52.331139 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.331067 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jl6g7\"" Jun 02 21:46:52.331139 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.331067 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 02 21:46:52.337881 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.337857 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt"] Jun 02 21:46:52.437833 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.437800 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h87t4\" (UniqueName: \"kubernetes.io/projected/eab6e1bd-38cb-4232-b18f-ccf04194dcc9-kube-api-access-h87t4\") pod \"dns-operator-controller-manager-648d5c98bc-ltmlt\" (UID: \"eab6e1bd-38cb-4232-b18f-ccf04194dcc9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:52.538934 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.538899 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h87t4\" (UniqueName: \"kubernetes.io/projected/eab6e1bd-38cb-4232-b18f-ccf04194dcc9-kube-api-access-h87t4\") pod \"dns-operator-controller-manager-648d5c98bc-ltmlt\" (UID: \"eab6e1bd-38cb-4232-b18f-ccf04194dcc9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:52.550680 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.550651 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h87t4\" (UniqueName: \"kubernetes.io/projected/eab6e1bd-38cb-4232-b18f-ccf04194dcc9-kube-api-access-h87t4\") pod \"dns-operator-controller-manager-648d5c98bc-ltmlt\" (UID: \"eab6e1bd-38cb-4232-b18f-ccf04194dcc9\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:52.639140 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.639042 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:52.775140 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:52.775111 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt"] Jun 02 21:46:52.777605 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:46:52.777578 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab6e1bd_38cb_4232_b18f_ccf04194dcc9.slice/crio-c75345e61fc8d8f6ac1f4aed4749a460dbd79871607491be8fa548989f68e276 WatchSource:0}: Error finding container c75345e61fc8d8f6ac1f4aed4749a460dbd79871607491be8fa548989f68e276: Status 404 returned error can't find the container with id c75345e61fc8d8f6ac1f4aed4749a460dbd79871607491be8fa548989f68e276 Jun 02 21:46:53.438535 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:53.438494 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" event={"ID":"eab6e1bd-38cb-4232-b18f-ccf04194dcc9","Type":"ContainerStarted","Data":"c75345e61fc8d8f6ac1f4aed4749a460dbd79871607491be8fa548989f68e276"} Jun 02 21:46:55.447960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:55.447923 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" event={"ID":"eab6e1bd-38cb-4232-b18f-ccf04194dcc9","Type":"ContainerStarted","Data":"e63b57c50d2b1e79cfbf91134a46ed70044a0263a62a86973e8d014c2f7d2bb4"} Jun 02 21:46:55.448389 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:55.447977 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:46:55.467458 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:55.467402 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" podStartSLOduration=1.478685568 podStartE2EDuration="3.467386801s" podCreationTimestamp="2026-06-02 21:46:52 +0000 UTC" firstStartedPulling="2026-06-02 21:46:52.779974281 +0000 UTC m=+440.581335236" lastFinishedPulling="2026-06-02 21:46:54.768675517 +0000 UTC m=+442.570036469" observedRunningTime="2026-06-02 21:46:55.466609036 +0000 UTC m=+443.267970048" watchObservedRunningTime="2026-06-02 21:46:55.467386801 +0000 UTC m=+443.268747775" Jun 02 21:46:57.305939 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.305896 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n"] Jun 02 21:46:57.310610 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.310567 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.314032 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.314009 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-f8gcd\"" Jun 02 21:46:57.319900 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.319870 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n"] Jun 02 21:46:57.375448 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.375410 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqvkp\" (UniqueName: \"kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.375622 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.375463 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.476022 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.475952 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jqvkp\" (UniqueName: \"kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.476200 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.476059 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.476462 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.476432 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.492487 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.492458 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqvkp\" (UniqueName: \"kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.623630 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.623534 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:46:57.784556 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:57.784523 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n"] Jun 02 21:46:57.787092 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:46:57.787061 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd35a2efc_0fcb_49b0_858b_2e2a3e304947.slice/crio-79339d65758431597f08cc9ade2cc534a8504aaa879418a40ac863b59c77c4ef WatchSource:0}: Error finding container 79339d65758431597f08cc9ade2cc534a8504aaa879418a40ac863b59c77c4ef: Status 404 returned error can't find the container with id 79339d65758431597f08cc9ade2cc534a8504aaa879418a40ac863b59c77c4ef Jun 02 21:46:58.460814 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:46:58.460772 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" event={"ID":"d35a2efc-0fcb-49b0-858b-2e2a3e304947","Type":"ContainerStarted","Data":"79339d65758431597f08cc9ade2cc534a8504aaa879418a40ac863b59c77c4ef"} Jun 02 21:47:03.482002 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:03.481965 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" event={"ID":"d35a2efc-0fcb-49b0-858b-2e2a3e304947","Type":"ContainerStarted","Data":"0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78"} Jun 02 21:47:03.482399 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:03.482078 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:47:03.502949 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:03.502895 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" podStartSLOduration=1.014931843 podStartE2EDuration="6.502879483s" podCreationTimestamp="2026-06-02 21:46:57 +0000 UTC" firstStartedPulling="2026-06-02 21:46:57.789967803 +0000 UTC m=+445.591328759" lastFinishedPulling="2026-06-02 21:47:03.277915446 +0000 UTC m=+451.079276399" observedRunningTime="2026-06-02 21:47:03.501416773 +0000 UTC m=+451.302777760" watchObservedRunningTime="2026-06-02 21:47:03.502879483 +0000 UTC m=+451.304240459" Jun 02 21:47:06.454827 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:06.454791 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-ltmlt" Jun 02 21:47:14.488686 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:14.488655 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:47:15.589229 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.587253 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r"] Jun 02 21:47:15.592235 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.592213 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.604474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.604448 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r"] Jun 02 21:47:15.621554 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.621518 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.621554 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.621558 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l2pl\" (UniqueName: \"kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.710865 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.710827 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r"] Jun 02 21:47:15.711096 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:47:15.711075 2558 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-5l2pl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" podUID="d4888f33-2e14-43d4-a93b-7f10c06b63a7" Jun 02 21:47:15.722354 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.722318 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.722463 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.722366 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5l2pl\" (UniqueName: \"kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.722746 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.722720 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:15.752946 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:15.752910 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l2pl\" (UniqueName: \"kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl\") pod \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:16.183822 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.183784 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n"] Jun 02 21:47:16.184063 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.184040 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" containerName="manager" containerID="cri-o://0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78" gracePeriod=2 Jun 02 21:47:16.189359 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.189330 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n"] Jun 02 21:47:16.202153 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.202128 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r"] Jun 02 21:47:16.207592 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.207550 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:16.208251 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.208228 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" containerName="manager" Jun 02 21:47:16.208385 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.208375 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" containerName="manager" Jun 02 21:47:16.208554 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.208538 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" containerName="manager" Jun 02 21:47:16.211815 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.211792 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r"] Jun 02 21:47:16.211959 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.211940 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.224313 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.224287 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:16.227417 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.227383 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.227565 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.227459 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhnl2\" (UniqueName: \"kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.243214 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.243171 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.267663 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.267631 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:16.271268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.271243 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.273523 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.273491 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.290300 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.290247 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:16.328928 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.328884 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.328928 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.328933 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nng6z\" (UniqueName: \"kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.329168 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.329018 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnl2\" (UniqueName: \"kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.329168 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.329108 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.329437 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.329420 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.341165 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.341128 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhnl2\" (UniqueName: \"kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-pq2tz\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.429745 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.429693 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.429745 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.429745 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nng6z\" (UniqueName: \"kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.430184 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.430158 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.443194 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.443172 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nng6z\" (UniqueName: \"kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z\") pod \"kuadrant-operator-controller-manager-84b657d985-8c2b4\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.450332 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.450311 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:47:16.452467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.452441 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.527092 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.527055 2558 generic.go:358] "Generic (PLEG): container finished" podID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" containerID="0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78" exitCode=0 Jun 02 21:47:16.527268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.527109 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" Jun 02 21:47:16.527268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.527163 2558 scope.go:117] "RemoveContainer" containerID="0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78" Jun 02 21:47:16.527268 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.527246 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:16.529484 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.529455 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.530356 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.530333 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume\") pod \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " Jun 02 21:47:16.530484 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.530398 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqvkp\" (UniqueName: \"kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp\") pod \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\" (UID: \"d35a2efc-0fcb-49b0-858b-2e2a3e304947\") " Jun 02 21:47:16.530818 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.530793 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d35a2efc-0fcb-49b0-858b-2e2a3e304947" (UID: "d35a2efc-0fcb-49b0-858b-2e2a3e304947"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:47:16.531331 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.531306 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.532348 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.532328 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:16.532774 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.532752 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp" (OuterVolumeSpecName: "kube-api-access-jqvkp") pod "d35a2efc-0fcb-49b0-858b-2e2a3e304947" (UID: "d35a2efc-0fcb-49b0-858b-2e2a3e304947"). InnerVolumeSpecName "kube-api-access-jqvkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:47:16.533036 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.532976 2558 status_manager.go:895] "Failed to get status for pod" podUID="d4888f33-2e14-43d4-a93b-7f10c06b63a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.534890 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.534858 2558 status_manager.go:895] "Failed to get status for pod" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-lms6n" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-lms6n\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.536115 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.536097 2558 scope.go:117] "RemoveContainer" containerID="0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78" Jun 02 21:47:16.536388 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:47:16.536364 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78\": container with ID starting with 0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78 not found: ID does not exist" containerID="0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78" Jun 02 21:47:16.536468 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.536403 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78"} err="failed to get container status \"0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78\": rpc error: code = NotFound desc = could not find container \"0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78\": container with ID starting with 0fcee9aa133d6d1862f0bad3dbd4aa44bc7904b880a64df5ec1057efe4c94d78 not found: ID does not exist" Jun 02 21:47:16.536518 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.536494 2558 status_manager.go:895] "Failed to get status for pod" podUID="d4888f33-2e14-43d4-a93b-7f10c06b63a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:16.568741 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.568710 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:16.582274 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.582246 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:16.631421 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.630926 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l2pl\" (UniqueName: \"kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl\") pod \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " Jun 02 21:47:16.631421 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.631020 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume\") pod \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\" (UID: \"d4888f33-2e14-43d4-a93b-7f10c06b63a7\") " Jun 02 21:47:16.631421 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.631300 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d35a2efc-0fcb-49b0-858b-2e2a3e304947-extensions-socket-volume\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:16.631421 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.631318 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqvkp\" (UniqueName: \"kubernetes.io/projected/d35a2efc-0fcb-49b0-858b-2e2a3e304947-kube-api-access-jqvkp\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:16.632051 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.631554 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d4888f33-2e14-43d4-a93b-7f10c06b63a7" (UID: "d4888f33-2e14-43d4-a93b-7f10c06b63a7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:47:16.634229 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.634197 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl" (OuterVolumeSpecName: "kube-api-access-5l2pl") pod "d4888f33-2e14-43d4-a93b-7f10c06b63a7" (UID: "d4888f33-2e14-43d4-a93b-7f10c06b63a7"). InnerVolumeSpecName "kube-api-access-5l2pl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:47:16.719867 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.719838 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:16.722691 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:47:16.722665 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c507f2_790b_4f2d_a952_cdda44c5c0e8.slice/crio-651f001faf24f0fdde41bbb48712025055fa164be0e2ddf56ffa26863fd5c6df WatchSource:0}: Error finding container 651f001faf24f0fdde41bbb48712025055fa164be0e2ddf56ffa26863fd5c6df: Status 404 returned error can't find the container with id 651f001faf24f0fdde41bbb48712025055fa164be0e2ddf56ffa26863fd5c6df Jun 02 21:47:16.732379 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.732317 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l2pl\" (UniqueName: \"kubernetes.io/projected/d4888f33-2e14-43d4-a93b-7f10c06b63a7-kube-api-access-5l2pl\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:16.732379 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.732337 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4888f33-2e14-43d4-a93b-7f10c06b63a7-extensions-socket-volume\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:16.750239 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.750126 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:16.758285 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:47:16.758259 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f1f7356_1423_4766_a74a_71adbe3f8692.slice/crio-9fe986945b7c8ab435c86671fe484dcf54d4ecb42c2fb63a6103edef87a0aaba WatchSource:0}: Error finding container 9fe986945b7c8ab435c86671fe484dcf54d4ecb42c2fb63a6103edef87a0aaba: Status 404 returned error can't find the container with id 9fe986945b7c8ab435c86671fe484dcf54d4ecb42c2fb63a6103edef87a0aaba Jun 02 21:47:16.812608 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.812578 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d35a2efc-0fcb-49b0-858b-2e2a3e304947" path="/var/lib/kubelet/pods/d35a2efc-0fcb-49b0-858b-2e2a3e304947/volumes" Jun 02 21:47:16.813215 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:16.813176 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4888f33-2e14-43d4-a93b-7f10c06b63a7" path="/var/lib/kubelet/pods/d4888f33-2e14-43d4-a93b-7f10c06b63a7/volumes" Jun 02 21:47:17.533563 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.533525 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" event={"ID":"34c507f2-790b-4f2d-a952-cdda44c5c0e8","Type":"ContainerStarted","Data":"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58"} Jun 02 21:47:17.533563 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.533566 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" event={"ID":"34c507f2-790b-4f2d-a952-cdda44c5c0e8","Type":"ContainerStarted","Data":"651f001faf24f0fdde41bbb48712025055fa164be0e2ddf56ffa26863fd5c6df"} Jun 02 21:47:17.533860 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.533609 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:17.535895 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.535872 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" Jun 02 21:47:17.536025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.535888 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" event={"ID":"6f1f7356-1423-4766-a74a-71adbe3f8692","Type":"ContainerStarted","Data":"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9"} Jun 02 21:47:17.536025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.535916 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" event={"ID":"6f1f7356-1423-4766-a74a-71adbe3f8692","Type":"ContainerStarted","Data":"9fe986945b7c8ab435c86671fe484dcf54d4ecb42c2fb63a6103edef87a0aaba"} Jun 02 21:47:17.536025 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.535945 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:17.562522 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.562455 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" podStartSLOduration=1.562439855 podStartE2EDuration="1.562439855s" podCreationTimestamp="2026-06-02 21:47:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:47:17.560081558 +0000 UTC m=+465.361442533" watchObservedRunningTime="2026-06-02 21:47:17.562439855 +0000 UTC m=+465.363800830" Jun 02 21:47:17.563209 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.563187 2558 status_manager.go:895] "Failed to get status for pod" podUID="d4888f33-2e14-43d4-a93b-7f10c06b63a7" pod="kuadrant-system/kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r" err="pods \"kuadrant-operator-controller-manager-6bc9f4c76f-w4s8r\" is forbidden: User \"system:node:ip-10-0-143-61.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-61.ec2.internal' and this object" Jun 02 21:47:17.590637 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:17.590580 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" podStartSLOduration=1.5905643839999999 podStartE2EDuration="1.590564384s" podCreationTimestamp="2026-06-02 21:47:16 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:47:17.589559057 +0000 UTC m=+465.390920032" watchObservedRunningTime="2026-06-02 21:47:17.590564384 +0000 UTC m=+465.391925358" Jun 02 21:47:28.541502 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.541467 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:28.541891 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.541564 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:28.626278 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.626245 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:28.626501 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.626474 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" podUID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" containerName="manager" containerID="cri-o://9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58" gracePeriod=10 Jun 02 21:47:28.877199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.877175 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:28.936143 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.936094 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhnl2\" (UniqueName: \"kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2\") pod \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " Jun 02 21:47:28.936301 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.936160 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume\") pod \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\" (UID: \"34c507f2-790b-4f2d-a952-cdda44c5c0e8\") " Jun 02 21:47:28.936546 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.936523 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c507f2-790b-4f2d-a952-cdda44c5c0e8" (UID: "34c507f2-790b-4f2d-a952-cdda44c5c0e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:47:28.938326 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:28.938299 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2" (OuterVolumeSpecName: "kube-api-access-nhnl2") pod "34c507f2-790b-4f2d-a952-cdda44c5c0e8" (UID: "34c507f2-790b-4f2d-a952-cdda44c5c0e8"). InnerVolumeSpecName "kube-api-access-nhnl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:47:29.037887 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.037851 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c507f2-790b-4f2d-a952-cdda44c5c0e8-extensions-socket-volume\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:29.037887 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.037881 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhnl2\" (UniqueName: \"kubernetes.io/projected/34c507f2-790b-4f2d-a952-cdda44c5c0e8-kube-api-access-nhnl2\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:29.579516 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.579473 2558 generic.go:358] "Generic (PLEG): container finished" podID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" containerID="9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58" exitCode=0 Jun 02 21:47:29.579957 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.579546 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" Jun 02 21:47:29.579957 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.579551 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" event={"ID":"34c507f2-790b-4f2d-a952-cdda44c5c0e8","Type":"ContainerDied","Data":"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58"} Jun 02 21:47:29.579957 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.579596 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz" event={"ID":"34c507f2-790b-4f2d-a952-cdda44c5c0e8","Type":"ContainerDied","Data":"651f001faf24f0fdde41bbb48712025055fa164be0e2ddf56ffa26863fd5c6df"} Jun 02 21:47:29.579957 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.579616 2558 scope.go:117] "RemoveContainer" containerID="9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58" Jun 02 21:47:29.588938 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.588917 2558 scope.go:117] "RemoveContainer" containerID="9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58" Jun 02 21:47:29.589257 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:47:29.589238 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58\": container with ID starting with 9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58 not found: ID does not exist" containerID="9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58" Jun 02 21:47:29.589302 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.589267 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58"} err="failed to get container status \"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58\": rpc error: code = NotFound desc = could not find container \"9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58\": container with ID starting with 9ef45b02c246e03a737253dbf04816a8ad6e56c38c59967c94badcae52da0a58 not found: ID does not exist" Jun 02 21:47:29.623369 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.623333 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:29.652411 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:29.652375 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-pq2tz"] Jun 02 21:47:30.811528 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:30.811492 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" path="/var/lib/kubelet/pods/34c507f2-790b-4f2d-a952-cdda44c5c0e8/volumes" Jun 02 21:47:47.228945 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.228913 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:47.229423 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.229175 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" podUID="6f1f7356-1423-4766-a74a-71adbe3f8692" containerName="manager" containerID="cri-o://75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9" gracePeriod=10 Jun 02 21:47:47.488195 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.488121 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:47.604719 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.604682 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume\") pod \"6f1f7356-1423-4766-a74a-71adbe3f8692\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " Jun 02 21:47:47.604887 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.604757 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nng6z\" (UniqueName: \"kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z\") pod \"6f1f7356-1423-4766-a74a-71adbe3f8692\" (UID: \"6f1f7356-1423-4766-a74a-71adbe3f8692\") " Jun 02 21:47:47.605127 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.605103 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6f1f7356-1423-4766-a74a-71adbe3f8692" (UID: "6f1f7356-1423-4766-a74a-71adbe3f8692"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:47:47.606815 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.606791 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z" (OuterVolumeSpecName: "kube-api-access-nng6z") pod "6f1f7356-1423-4766-a74a-71adbe3f8692" (UID: "6f1f7356-1423-4766-a74a-71adbe3f8692"). InnerVolumeSpecName "kube-api-access-nng6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:47:47.645140 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.645100 2558 generic.go:358] "Generic (PLEG): container finished" podID="6f1f7356-1423-4766-a74a-71adbe3f8692" containerID="75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9" exitCode=0 Jun 02 21:47:47.645309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.645148 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" event={"ID":"6f1f7356-1423-4766-a74a-71adbe3f8692","Type":"ContainerDied","Data":"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9"} Jun 02 21:47:47.645309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.645171 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" Jun 02 21:47:47.645309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.645176 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4" event={"ID":"6f1f7356-1423-4766-a74a-71adbe3f8692","Type":"ContainerDied","Data":"9fe986945b7c8ab435c86671fe484dcf54d4ecb42c2fb63a6103edef87a0aaba"} Jun 02 21:47:47.645309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.645196 2558 scope.go:117] "RemoveContainer" containerID="75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9" Jun 02 21:47:47.654352 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.654326 2558 scope.go:117] "RemoveContainer" containerID="75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9" Jun 02 21:47:47.654607 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:47:47.654585 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9\": container with ID starting with 75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9 not found: ID does not exist" containerID="75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9" Jun 02 21:47:47.654654 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.654617 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9"} err="failed to get container status \"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9\": rpc error: code = NotFound desc = could not find container \"75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9\": container with ID starting with 75194dce361b116b214f010db22e940d4d771ff7e537b0622010f651de36c5f9 not found: ID does not exist" Jun 02 21:47:47.670160 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.670126 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:47.674171 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.674138 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-8c2b4"] Jun 02 21:47:47.705847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.705804 2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f1f7356-1423-4766-a74a-71adbe3f8692-extensions-socket-volume\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:47.705847 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:47.705842 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nng6z\" (UniqueName: \"kubernetes.io/projected/6f1f7356-1423-4766-a74a-71adbe3f8692-kube-api-access-nng6z\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:47:48.811002 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:47:48.810961 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f1f7356-1423-4766-a74a-71adbe3f8692" path="/var/lib/kubelet/pods/6f1f7356-1423-4766-a74a-71adbe3f8692/volumes" Jun 02 21:48:05.313595 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313552 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313874 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" containerName="manager" Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313884 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" containerName="manager" Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313897 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f1f7356-1423-4766-a74a-71adbe3f8692" containerName="manager" Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313902 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f1f7356-1423-4766-a74a-71adbe3f8692" containerName="manager" Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313964 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="6f1f7356-1423-4766-a74a-71adbe3f8692" containerName="manager" Jun 02 21:48:05.314103 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.313977 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="34c507f2-790b-4f2d-a952-cdda44c5c0e8" containerName="manager" Jun 02 21:48:05.316969 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.316943 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.320449 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.320423 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 02 21:48:05.320560 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.320423 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vwmsl\"" Jun 02 21:48:05.328205 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.328180 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:05.347598 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.347561 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsctq\" (UniqueName: \"kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.347754 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.347616 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.405253 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.405220 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:05.448522 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.448485 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hsctq\" (UniqueName: \"kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.448695 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.448616 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.449314 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.449293 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.457454 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.457430 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsctq\" (UniqueName: \"kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq\") pod \"limitador-limitador-7d549b5b-tbw9l\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.628258 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.628163 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:05.759399 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.759367 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:48:05.763772 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.763748 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.772909 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.772881 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:48:05.778618 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.778542 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:05.782885 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:48:05.782853 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1abb2a30_1039_4b68_9fd9_1a602c5bb76d.slice/crio-eb32c85441dd6743447285fb64f27baf5fc1eb7dc779f2326bbe00453b68ccbe WatchSource:0}: Error finding container eb32c85441dd6743447285fb64f27baf5fc1eb7dc779f2326bbe00453b68ccbe: Status 404 returned error can't find the container with id eb32c85441dd6743447285fb64f27baf5fc1eb7dc779f2326bbe00453b68ccbe Jun 02 21:48:05.802667 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.802641 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:48:05.850936 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.850894 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shll\" (UniqueName: \"kubernetes.io/projected/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-kube-api-access-5shll\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.851173 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.851125 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-config-file\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.951769 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.951724 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5shll\" (UniqueName: \"kubernetes.io/projected/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-kube-api-access-5shll\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.951953 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.951820 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-config-file\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.952418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.952401 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-config-file\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:05.960585 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:05.960558 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shll\" (UniqueName: \"kubernetes.io/projected/9d288fe2-7a8b-4e3d-a7dc-dbdae5286683-kube-api-access-5shll\") pod \"limitador-limitador-78c99df468-fcfmz\" (UID: \"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683\") " pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:06.075364 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:06.075322 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:06.220275 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:06.220249 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:48:06.222817 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:48:06.222783 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d288fe2_7a8b_4e3d_a7dc_dbdae5286683.slice/crio-7802c6ff613944efb0d41c83e2f91717671e3ab7d34ac0332c990bed9a3425c3 WatchSource:0}: Error finding container 7802c6ff613944efb0d41c83e2f91717671e3ab7d34ac0332c990bed9a3425c3: Status 404 returned error can't find the container with id 7802c6ff613944efb0d41c83e2f91717671e3ab7d34ac0332c990bed9a3425c3 Jun 02 21:48:06.735579 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:06.735494 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" event={"ID":"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683","Type":"ContainerStarted","Data":"7802c6ff613944efb0d41c83e2f91717671e3ab7d34ac0332c990bed9a3425c3"} Jun 02 21:48:06.737878 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:06.737780 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" event={"ID":"1abb2a30-1039-4b68-9fd9-1a602c5bb76d","Type":"ContainerStarted","Data":"eb32c85441dd6743447285fb64f27baf5fc1eb7dc779f2326bbe00453b68ccbe"} Jun 02 21:48:09.749513 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.749474 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" event={"ID":"9d288fe2-7a8b-4e3d-a7dc-dbdae5286683","Type":"ContainerStarted","Data":"8cee10eeb12fc3113405ca2715742cc57b85898477740122b589ff9d5ec2c003"} Jun 02 21:48:09.749942 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.749576 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:09.750929 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.750903 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" event={"ID":"1abb2a30-1039-4b68-9fd9-1a602c5bb76d","Type":"ContainerStarted","Data":"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e"} Jun 02 21:48:09.751039 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.751023 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:09.774487 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.774428 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" podStartSLOduration=2.147546845 podStartE2EDuration="4.774412274s" podCreationTimestamp="2026-06-02 21:48:05 +0000 UTC" firstStartedPulling="2026-06-02 21:48:06.224876223 +0000 UTC m=+514.026237175" lastFinishedPulling="2026-06-02 21:48:08.851741641 +0000 UTC m=+516.653102604" observedRunningTime="2026-06-02 21:48:09.772633149 +0000 UTC m=+517.573994136" watchObservedRunningTime="2026-06-02 21:48:09.774412274 +0000 UTC m=+517.575773249" Jun 02 21:48:09.794370 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:09.794316 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" podStartSLOduration=1.726684706 podStartE2EDuration="4.794297017s" podCreationTimestamp="2026-06-02 21:48:05 +0000 UTC" firstStartedPulling="2026-06-02 21:48:05.79208541 +0000 UTC m=+513.593446366" lastFinishedPulling="2026-06-02 21:48:08.859697724 +0000 UTC m=+516.661058677" observedRunningTime="2026-06-02 21:48:09.791505039 +0000 UTC m=+517.592866013" watchObservedRunningTime="2026-06-02 21:48:09.794297017 +0000 UTC m=+517.595657993" Jun 02 21:48:20.755063 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:20.755028 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-fcfmz" Jun 02 21:48:20.755474 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:20.755086 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:20.812762 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:20.812726 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:20.812958 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:20.812933 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" podUID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" containerName="limitador" containerID="cri-o://d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e" gracePeriod=30 Jun 02 21:48:21.767660 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.767632 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:21.794861 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.794827 2558 generic.go:358] "Generic (PLEG): container finished" podID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" containerID="d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e" exitCode=0 Jun 02 21:48:21.795049 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.794893 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" Jun 02 21:48:21.795049 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.794908 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" event={"ID":"1abb2a30-1039-4b68-9fd9-1a602c5bb76d","Type":"ContainerDied","Data":"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e"} Jun 02 21:48:21.795049 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.794954 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-tbw9l" event={"ID":"1abb2a30-1039-4b68-9fd9-1a602c5bb76d","Type":"ContainerDied","Data":"eb32c85441dd6743447285fb64f27baf5fc1eb7dc779f2326bbe00453b68ccbe"} Jun 02 21:48:21.795049 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.794970 2558 scope.go:117] "RemoveContainer" containerID="d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e" Jun 02 21:48:21.803226 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.803201 2558 scope.go:117] "RemoveContainer" containerID="d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e" Jun 02 21:48:21.803511 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:48:21.803486 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e\": container with ID starting with d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e not found: ID does not exist" containerID="d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e" Jun 02 21:48:21.803616 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.803521 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e"} err="failed to get container status \"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e\": rpc error: code = NotFound desc = could not find container \"d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e\": container with ID starting with d966b772a7926ef672342f80fc4db1064626ac1dde05ea3b25acd2ce5de7c35e not found: ID does not exist" Jun 02 21:48:21.898798 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.898765 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsctq\" (UniqueName: \"kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq\") pod \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " Jun 02 21:48:21.898964 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.898815 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file\") pod \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\" (UID: \"1abb2a30-1039-4b68-9fd9-1a602c5bb76d\") " Jun 02 21:48:21.899252 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.899223 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file" (OuterVolumeSpecName: "config-file") pod "1abb2a30-1039-4b68-9fd9-1a602c5bb76d" (UID: "1abb2a30-1039-4b68-9fd9-1a602c5bb76d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:48:21.900924 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.900901 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq" (OuterVolumeSpecName: "kube-api-access-hsctq") pod "1abb2a30-1039-4b68-9fd9-1a602c5bb76d" (UID: "1abb2a30-1039-4b68-9fd9-1a602c5bb76d"). InnerVolumeSpecName "kube-api-access-hsctq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:48:22.000012 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.999909 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsctq\" (UniqueName: \"kubernetes.io/projected/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-kube-api-access-hsctq\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:48:22.000012 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:21.999941 2558 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1abb2a30-1039-4b68-9fd9-1a602c5bb76d-config-file\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:48:22.115459 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:22.115428 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:22.118679 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:22.118649 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-tbw9l"] Jun 02 21:48:22.811821 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:22.811790 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" path="/var/lib/kubelet/pods/1abb2a30-1039-4b68-9fd9-1a602c5bb76d/volumes" Jun 02 21:48:25.820072 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.820036 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9r9wl"] Jun 02 21:48:25.820591 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.820569 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" containerName="limitador" Jun 02 21:48:25.820690 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.820594 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" containerName="limitador" Jun 02 21:48:25.820690 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.820670 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="1abb2a30-1039-4b68-9fd9-1a602c5bb76d" containerName="limitador" Jun 02 21:48:25.823630 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.823608 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:25.827835 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.827812 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 02 21:48:25.842750 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.842722 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9r9wl"] Jun 02 21:48:25.934323 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.934279 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n78g\" (UniqueName: \"kubernetes.io/projected/f199ed9f-72ea-412c-8fa9-446ac6ecd810-kube-api-access-7n78g\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:25.934477 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:25.934410 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f199ed9f-72ea-412c-8fa9-446ac6ecd810-data\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.035195 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.035151 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7n78g\" (UniqueName: \"kubernetes.io/projected/f199ed9f-72ea-412c-8fa9-446ac6ecd810-kube-api-access-7n78g\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.035411 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.035240 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f199ed9f-72ea-412c-8fa9-446ac6ecd810-data\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.035628 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.035606 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f199ed9f-72ea-412c-8fa9-446ac6ecd810-data\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.044063 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.044035 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n78g\" (UniqueName: \"kubernetes.io/projected/f199ed9f-72ea-412c-8fa9-446ac6ecd810-kube-api-access-7n78g\") pod \"postgres-68c4fbbd6f-9r9wl\" (UID: \"f199ed9f-72ea-412c-8fa9-446ac6ecd810\") " pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.134917 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.134813 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:26.273317 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.273287 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-9r9wl"] Jun 02 21:48:26.276120 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:48:26.276088 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf199ed9f_72ea_412c_8fa9_446ac6ecd810.slice/crio-a968d6963966f9cc112023df5d90b5424d3a46fd0d2d9c213258895b44035cfc WatchSource:0}: Error finding container a968d6963966f9cc112023df5d90b5424d3a46fd0d2d9c213258895b44035cfc: Status 404 returned error can't find the container with id a968d6963966f9cc112023df5d90b5424d3a46fd0d2d9c213258895b44035cfc Jun 02 21:48:26.812144 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:26.812106 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" event={"ID":"f199ed9f-72ea-412c-8fa9-446ac6ecd810","Type":"ContainerStarted","Data":"a968d6963966f9cc112023df5d90b5424d3a46fd0d2d9c213258895b44035cfc"} Jun 02 21:48:28.820354 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:28.820259 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" event={"ID":"f199ed9f-72ea-412c-8fa9-446ac6ecd810","Type":"ContainerStarted","Data":"7f4a5a1bd84569e357e7a50d40632e61e7ddc634f06b0af6fe0f418c1aeb9b81"} Jun 02 21:48:28.820354 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:28.820348 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:28.835888 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:28.835825 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" podStartSLOduration=1.571189492 podStartE2EDuration="3.83580428s" podCreationTimestamp="2026-06-02 21:48:25 +0000 UTC" firstStartedPulling="2026-06-02 21:48:26.277322952 +0000 UTC m=+534.078683905" lastFinishedPulling="2026-06-02 21:48:28.541937736 +0000 UTC m=+536.343298693" observedRunningTime="2026-06-02 21:48:28.835206992 +0000 UTC m=+536.636567968" watchObservedRunningTime="2026-06-02 21:48:28.83580428 +0000 UTC m=+536.637165256" Jun 02 21:48:34.857200 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:34.857165 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-9r9wl" Jun 02 21:48:59.430338 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.430303 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:48:59.439269 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.439238 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:48:59.441615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.441580 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:48:59.441799 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.441720 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:48:59.441799 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.441785 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:48:59.441951 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.441825 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:48:59.441951 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.441882 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-k2xhk\"" Jun 02 21:48:59.502253 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.502213 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw8r9\" (UniqueName: \"kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9\") pod \"maas-keycloak-0\" (UID: \"85073f1b-ffff-4cac-a0d6-37eb67496190\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:48:59.603479 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.603438 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xw8r9\" (UniqueName: \"kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9\") pod \"maas-keycloak-0\" (UID: \"85073f1b-ffff-4cac-a0d6-37eb67496190\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:48:59.612458 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.612427 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw8r9\" (UniqueName: \"kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9\") pod \"maas-keycloak-0\" (UID: \"85073f1b-ffff-4cac-a0d6-37eb67496190\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:48:59.750122 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.750026 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:48:59.886849 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.886822 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:48:59.889939 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:48:59.889904 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85073f1b_ffff_4cac_a0d6_37eb67496190.slice/crio-be35dd7d26602434241f677fa34c796d8d3870f0330fa4400e29da940b80425c WatchSource:0}: Error finding container be35dd7d26602434241f677fa34c796d8d3870f0330fa4400e29da940b80425c: Status 404 returned error can't find the container with id be35dd7d26602434241f677fa34c796d8d3870f0330fa4400e29da940b80425c Jun 02 21:48:59.928346 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:48:59.928299 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"85073f1b-ffff-4cac-a0d6-37eb67496190","Type":"ContainerStarted","Data":"be35dd7d26602434241f677fa34c796d8d3870f0330fa4400e29da940b80425c"} Jun 02 21:49:04.950658 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:04.950605 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"85073f1b-ffff-4cac-a0d6-37eb67496190","Type":"ContainerStarted","Data":"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3"} Jun 02 21:49:04.970310 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:04.970190 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.227529523 podStartE2EDuration="5.970169708s" podCreationTimestamp="2026-06-02 21:48:59 +0000 UTC" firstStartedPulling="2026-06-02 21:48:59.891617972 +0000 UTC m=+567.692978925" lastFinishedPulling="2026-06-02 21:49:04.634258152 +0000 UTC m=+572.435619110" observedRunningTime="2026-06-02 21:49:04.968754038 +0000 UTC m=+572.770115019" watchObservedRunningTime="2026-06-02 21:49:04.970169708 +0000 UTC m=+572.771530685" Jun 02 21:49:05.751188 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:05.751137 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:49:05.753171 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:05.753130 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:06.750773 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:06.750718 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:07.751179 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:07.751134 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:08.750617 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:08.750557 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:09.751198 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:09.751154 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:49:09.752486 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:09.752452 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:10.751218 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:10.751170 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:11.751434 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:11.751377 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:12.750734 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:12.750686 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:13.751281 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:13.751230 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:14.751368 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:14.751316 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:15.750826 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:15.750771 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:16.750918 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:16.750868 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:17.751416 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:17.751370 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.36:9000/health/started\": dial tcp 10.132.0.36:9000: connect: connection refused" Jun 02 21:49:18.892614 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:18.892570 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:49:18.912971 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:18.912912 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:49:28.899438 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:28.899396 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:49:32.739754 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:32.739720 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:49:32.740249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:32.740233 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:49:35.184870 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.184826 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:35.250265 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.250236 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:35.250425 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.250349 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:35.253242 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.253208 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 21:49:35.253383 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.253362 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8klnz\"" Jun 02 21:49:35.441658 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.441568 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vzh\" (UniqueName: \"kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh\") pod \"maas-controller-778865fb68-nzwzn\" (UID: \"2c9284c2-d8d8-4d82-a1de-717655c9a471\") " pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:35.542922 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.542878 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49vzh\" (UniqueName: \"kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh\") pod \"maas-controller-778865fb68-nzwzn\" (UID: \"2c9284c2-d8d8-4d82-a1de-717655c9a471\") " pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:35.551791 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.551760 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vzh\" (UniqueName: \"kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh\") pod \"maas-controller-778865fb68-nzwzn\" (UID: \"2c9284c2-d8d8-4d82-a1de-717655c9a471\") " pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:35.560511 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.560485 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:35.693555 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:35.693527 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:35.695448 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:49:35.695405 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c9284c2_d8d8_4d82_a1de_717655c9a471.slice/crio-d83f06c3373a8b33a594c08de13649b82bb454498e1f11bb17614a5c69465b92 WatchSource:0}: Error finding container d83f06c3373a8b33a594c08de13649b82bb454498e1f11bb17614a5c69465b92: Status 404 returned error can't find the container with id d83f06c3373a8b33a594c08de13649b82bb454498e1f11bb17614a5c69465b92 Jun 02 21:49:36.076947 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:36.076850 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778865fb68-nzwzn" event={"ID":"2c9284c2-d8d8-4d82-a1de-717655c9a471","Type":"ContainerStarted","Data":"d83f06c3373a8b33a594c08de13649b82bb454498e1f11bb17614a5c69465b92"} Jun 02 21:49:39.089536 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:39.089489 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778865fb68-nzwzn" event={"ID":"2c9284c2-d8d8-4d82-a1de-717655c9a471","Type":"ContainerStarted","Data":"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce"} Jun 02 21:49:39.089957 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:39.089555 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:39.111608 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:39.111548 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-778865fb68-nzwzn" podStartSLOduration=1.438706957 podStartE2EDuration="4.111531855s" podCreationTimestamp="2026-06-02 21:49:35 +0000 UTC" firstStartedPulling="2026-06-02 21:49:35.69683725 +0000 UTC m=+603.498198204" lastFinishedPulling="2026-06-02 21:49:38.369662148 +0000 UTC m=+606.171023102" observedRunningTime="2026-06-02 21:49:39.110022571 +0000 UTC m=+606.911383547" watchObservedRunningTime="2026-06-02 21:49:39.111531855 +0000 UTC m=+606.912892829" Jun 02 21:49:49.693014 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:49.692962 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:49.693610 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:49.693314 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-778865fb68-nzwzn" podUID="2c9284c2-d8d8-4d82-a1de-717655c9a471" containerName="manager" containerID="cri-o://fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce" gracePeriod=30 Jun 02 21:49:49.711131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:49.711096 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:49.941251 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:49.941226 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:50.062184 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.062087 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49vzh\" (UniqueName: \"kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh\") pod \"2c9284c2-d8d8-4d82-a1de-717655c9a471\" (UID: \"2c9284c2-d8d8-4d82-a1de-717655c9a471\") " Jun 02 21:49:50.064369 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.064344 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh" (OuterVolumeSpecName: "kube-api-access-49vzh") pod "2c9284c2-d8d8-4d82-a1de-717655c9a471" (UID: "2c9284c2-d8d8-4d82-a1de-717655c9a471"). InnerVolumeSpecName "kube-api-access-49vzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:49:50.130976 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.130941 2558 generic.go:358] "Generic (PLEG): container finished" podID="2c9284c2-d8d8-4d82-a1de-717655c9a471" containerID="fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce" exitCode=0 Jun 02 21:49:50.131162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.131013 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-778865fb68-nzwzn" Jun 02 21:49:50.131162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.131025 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778865fb68-nzwzn" event={"ID":"2c9284c2-d8d8-4d82-a1de-717655c9a471","Type":"ContainerDied","Data":"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce"} Jun 02 21:49:50.131162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.131068 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-778865fb68-nzwzn" event={"ID":"2c9284c2-d8d8-4d82-a1de-717655c9a471","Type":"ContainerDied","Data":"d83f06c3373a8b33a594c08de13649b82bb454498e1f11bb17614a5c69465b92"} Jun 02 21:49:50.131162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.131083 2558 scope.go:117] "RemoveContainer" containerID="fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce" Jun 02 21:49:50.139923 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.139905 2558 scope.go:117] "RemoveContainer" containerID="fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce" Jun 02 21:49:50.140192 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:49:50.140176 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce\": container with ID starting with fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce not found: ID does not exist" containerID="fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce" Jun 02 21:49:50.140246 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.140200 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce"} err="failed to get container status \"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce\": rpc error: code = NotFound desc = could not find container \"fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce\": container with ID starting with fcb8e41c895b0f2d4431f4a529514359811056ef785865d22a73207788ad35ce not found: ID does not exist" Jun 02 21:49:50.157467 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.157436 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:50.158960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.158938 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-778865fb68-nzwzn"] Jun 02 21:49:50.163768 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.163737 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49vzh\" (UniqueName: \"kubernetes.io/projected/2c9284c2-d8d8-4d82-a1de-717655c9a471-kube-api-access-49vzh\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:49:50.810650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:49:50.810617 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c9284c2-d8d8-4d82-a1de-717655c9a471" path="/var/lib/kubelet/pods/2c9284c2-d8d8-4d82-a1de-717655c9a471/volumes" Jun 02 21:50:09.297738 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.297701 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-85d8c9f8f6-6nt7d"] Jun 02 21:50:09.303249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.298068 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c9284c2-d8d8-4d82-a1de-717655c9a471" containerName="manager" Jun 02 21:50:09.303249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.298079 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c9284c2-d8d8-4d82-a1de-717655c9a471" containerName="manager" Jun 02 21:50:09.303249 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.298126 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="2c9284c2-d8d8-4d82-a1de-717655c9a471" containerName="manager" Jun 02 21:50:09.303494 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.303477 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.305883 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.305865 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 21:50:09.311833 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.311808 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-85d8c9f8f6-6nt7d"] Jun 02 21:50:09.435838 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.435801 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.435838 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.435843 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8hh2\" (UniqueName: \"kubernetes.io/projected/7ab29c97-52a0-4347-abad-6b4f7e7027f4-kube-api-access-v8hh2\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.537211 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.537173 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.537497 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.537472 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v8hh2\" (UniqueName: \"kubernetes.io/projected/7ab29c97-52a0-4347-abad-6b4f7e7027f4-kube-api-access-v8hh2\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.537783 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:50:09.537327 2558 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 02 21:50:09.537899 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:50:09.537882 2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls podName:7ab29c97-52a0-4347-abad-6b4f7e7027f4 nodeName:}" failed. No retries permitted until 2026-06-02 21:50:10.037852005 +0000 UTC m=+637.839212979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls") pod "maas-api-85d8c9f8f6-6nt7d" (UID: "7ab29c97-52a0-4347-abad-6b4f7e7027f4") : secret "maas-api-serving-cert" not found Jun 02 21:50:09.546712 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.546681 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8hh2\" (UniqueName: \"kubernetes.io/projected/7ab29c97-52a0-4347-abad-6b4f7e7027f4-kube-api-access-v8hh2\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:09.852176 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:09.852146 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:50:10.042131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:10.042095 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:10.044541 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:10.044520 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ab29c97-52a0-4347-abad-6b4f7e7027f4-maas-api-tls\") pod \"maas-api-85d8c9f8f6-6nt7d\" (UID: \"7ab29c97-52a0-4347-abad-6b4f7e7027f4\") " pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:10.214480 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:10.214442 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:10.350927 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:10.350901 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-85d8c9f8f6-6nt7d"] Jun 02 21:50:10.353782 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:50:10.353741 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ab29c97_52a0_4347_abad_6b4f7e7027f4.slice/crio-3fd04c3deb6a8d21fa8b8e0062fc69b3b23c8b813527c0686954df2558c22e97 WatchSource:0}: Error finding container 3fd04c3deb6a8d21fa8b8e0062fc69b3b23c8b813527c0686954df2558c22e97: Status 404 returned error can't find the container with id 3fd04c3deb6a8d21fa8b8e0062fc69b3b23c8b813527c0686954df2558c22e97 Jun 02 21:50:10.354951 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:10.354934 2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:50:11.204921 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:11.204881 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" event={"ID":"7ab29c97-52a0-4347-abad-6b4f7e7027f4","Type":"ContainerStarted","Data":"3fd04c3deb6a8d21fa8b8e0062fc69b3b23c8b813527c0686954df2558c22e97"} Jun 02 21:50:12.210511 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:12.210475 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" event={"ID":"7ab29c97-52a0-4347-abad-6b4f7e7027f4","Type":"ContainerStarted","Data":"2e6fac5dcd87504a1030fb864906dd985a0a490566e9ccf5154445d549ba07bf"} Jun 02 21:50:12.210968 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:12.210599 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:12.252189 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:12.252136 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" podStartSLOduration=1.9000890419999998 podStartE2EDuration="3.252119462s" podCreationTimestamp="2026-06-02 21:50:09 +0000 UTC" firstStartedPulling="2026-06-02 21:50:10.355099543 +0000 UTC m=+638.156460499" lastFinishedPulling="2026-06-02 21:50:11.707129965 +0000 UTC m=+639.508490919" observedRunningTime="2026-06-02 21:50:12.248913507 +0000 UTC m=+640.050274482" watchObservedRunningTime="2026-06-02 21:50:12.252119462 +0000 UTC m=+640.053480416" Jun 02 21:50:18.219560 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:18.219529 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-85d8c9f8f6-6nt7d" Jun 02 21:50:20.124159 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:20.124119 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:20.124581 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:20.124412 2558 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" containerID="cri-o://6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3" gracePeriod=30 Jun 02 21:50:22.169959 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.169934 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.246405 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.246372 2558 generic.go:358] "Generic (PLEG): container finished" podID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerID="6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3" exitCode=143 Jun 02 21:50:22.246578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.246441 2558 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.246578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.246453 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"85073f1b-ffff-4cac-a0d6-37eb67496190","Type":"ContainerDied","Data":"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3"} Jun 02 21:50:22.246578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.246492 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"85073f1b-ffff-4cac-a0d6-37eb67496190","Type":"ContainerDied","Data":"be35dd7d26602434241f677fa34c796d8d3870f0330fa4400e29da940b80425c"} Jun 02 21:50:22.246578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.246512 2558 scope.go:117] "RemoveContainer" containerID="6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3" Jun 02 21:50:22.255590 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.255563 2558 scope.go:117] "RemoveContainer" containerID="6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3" Jun 02 21:50:22.255912 ip-10-0-143-61 kubenswrapper[2558]: E0602 21:50:22.255886 2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3\": container with ID starting with 6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3 not found: ID does not exist" containerID="6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3" Jun 02 21:50:22.256034 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.255924 2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3"} err="failed to get container status \"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3\": rpc error: code = NotFound desc = could not find container \"6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3\": container with ID starting with 6539cba8b21148c25b92c21d2bde9a2121613e3aee88aad4911c05db8fee13f3 not found: ID does not exist" Jun 02 21:50:22.350405 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.350317 2558 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw8r9\" (UniqueName: \"kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9\") pod \"85073f1b-ffff-4cac-a0d6-37eb67496190\" (UID: \"85073f1b-ffff-4cac-a0d6-37eb67496190\") " Jun 02 21:50:22.352518 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.352492 2558 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9" (OuterVolumeSpecName: "kube-api-access-xw8r9") pod "85073f1b-ffff-4cac-a0d6-37eb67496190" (UID: "85073f1b-ffff-4cac-a0d6-37eb67496190"). InnerVolumeSpecName "kube-api-access-xw8r9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:50:22.451574 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.451523 2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw8r9\" (UniqueName: \"kubernetes.io/projected/85073f1b-ffff-4cac-a0d6-37eb67496190-kube-api-access-xw8r9\") on node \"ip-10-0-143-61.ec2.internal\" DevicePath \"\"" Jun 02 21:50:22.568965 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.568929 2558 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:22.574192 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.574162 2558 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:22.594222 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.594189 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:22.594568 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.594555 2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" Jun 02 21:50:22.594611 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.594570 2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" Jun 02 21:50:22.594648 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.594628 2558 memory_manager.go:356] "RemoveStaleState removing state" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" containerName="keycloak" Jun 02 21:50:22.598823 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.598805 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.601341 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.601276 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:50:22.601341 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.601294 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-k2xhk\"" Jun 02 21:50:22.601341 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.601294 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 21:50:22.601341 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.601324 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:50:22.601617 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.601299 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:50:22.606199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.606058 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:22.754558 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.754521 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wxtr\" (UniqueName: \"kubernetes.io/projected/f9626154-4fe8-46d2-9dd0-5237f47d65d9-kube-api-access-4wxtr\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.754724 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.754576 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f9626154-4fe8-46d2-9dd0-5237f47d65d9-test-realms\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.811487 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.811452 2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85073f1b-ffff-4cac-a0d6-37eb67496190" path="/var/lib/kubelet/pods/85073f1b-ffff-4cac-a0d6-37eb67496190/volumes" Jun 02 21:50:22.855157 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.855072 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4wxtr\" (UniqueName: \"kubernetes.io/projected/f9626154-4fe8-46d2-9dd0-5237f47d65d9-kube-api-access-4wxtr\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.855157 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.855130 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f9626154-4fe8-46d2-9dd0-5237f47d65d9-test-realms\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.855774 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.855755 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/f9626154-4fe8-46d2-9dd0-5237f47d65d9-test-realms\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.862684 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.862654 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wxtr\" (UniqueName: \"kubernetes.io/projected/f9626154-4fe8-46d2-9dd0-5237f47d65d9-kube-api-access-4wxtr\") pod \"maas-keycloak-0\" (UID: \"f9626154-4fe8-46d2-9dd0-5237f47d65d9\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:22.909356 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:22.909331 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:23.044187 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:23.044159 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:50:23.048280 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:50:23.048245 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9626154_4fe8_46d2_9dd0_5237f47d65d9.slice/crio-b85190a514089b8e4c078c538025486ede4de12a6be08c727fbda99329bcf241 WatchSource:0}: Error finding container b85190a514089b8e4c078c538025486ede4de12a6be08c727fbda99329bcf241: Status 404 returned error can't find the container with id b85190a514089b8e4c078c538025486ede4de12a6be08c727fbda99329bcf241 Jun 02 21:50:23.251830 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:23.251796 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f9626154-4fe8-46d2-9dd0-5237f47d65d9","Type":"ContainerStarted","Data":"b85190a514089b8e4c078c538025486ede4de12a6be08c727fbda99329bcf241"} Jun 02 21:50:24.258313 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:24.258270 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f9626154-4fe8-46d2-9dd0-5237f47d65d9","Type":"ContainerStarted","Data":"c40d8fb6de76304a0f334c2df83d92a755779168578018aedbf7e2c116717761"} Jun 02 21:50:24.277154 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:24.277099 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.93840557 podStartE2EDuration="2.277079433s" podCreationTimestamp="2026-06-02 21:50:22 +0000 UTC" firstStartedPulling="2026-06-02 21:50:23.050057136 +0000 UTC m=+650.851418089" lastFinishedPulling="2026-06-02 21:50:23.388730996 +0000 UTC m=+651.190091952" observedRunningTime="2026-06-02 21:50:24.274717694 +0000 UTC m=+652.076078670" watchObservedRunningTime="2026-06-02 21:50:24.277079433 +0000 UTC m=+652.078440408" Jun 02 21:50:24.910521 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:24.910463 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:24.912005 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:24.911939 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:25.910783 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:25.910736 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:26.910855 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:26.910804 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:27.910100 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:27.910046 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:28.910671 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:28.910598 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:29.910446 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:29.910355 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:30.910356 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:30.910300 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:31.910004 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:31.909938 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:32.909454 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:32.909395 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:32.910130 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:32.910096 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:33.910098 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:33.910055 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:34.909916 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:34.909861 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:35.910784 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:35.910736 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:36.910386 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:36.910337 2558 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.39:9000/health/started\": dial tcp 10.132.0.39:9000: connect: connection refused" Jun 02 21:50:38.022207 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:38.022170 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:50:38.038389 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:38.038326 2558 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f9626154-4fe8-46d2-9dd0-5237f47d65d9" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:50:48.028496 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:50:48.028461 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:51:21.687718 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.687683 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r"] Jun 02 21:51:21.692544 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.692516 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.695971 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.695947 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qmdwv\"" Jun 02 21:51:21.696141 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.696018 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 02 21:51:21.696141 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.695969 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 21:51:21.696141 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.695975 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 21:51:21.702197 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.702168 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r"] Jun 02 21:51:21.806418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806382 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs829\" (UniqueName: \"kubernetes.io/projected/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kube-api-access-bs829\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.806588 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806447 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.806701 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806681 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.806744 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806722 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.806782 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806750 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.806782 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.806769 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.907770 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.907729 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.907942 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.907792 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908028 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.907982 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bs829\" (UniqueName: \"kubernetes.io/projected/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kube-api-access-bs829\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908088 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908063 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908173 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908156 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908326 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908308 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908468 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-model-cache\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908220 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.908750 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.908718 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-home\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.910518 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.910491 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-dshm\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.910637 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.910582 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:21.916510 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:21.916482 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs829\" (UniqueName: \"kubernetes.io/projected/63bd85c3-8a0d-44a1-99cd-0e1e727f2c90-kube-api-access-bs829\") pod \"e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r\" (UID: \"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90\") " pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:22.005274 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:22.005149 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:22.122753 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:22.122718 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:51:22.142519 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:22.142438 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r"] Jun 02 21:51:22.145195 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:51:22.145166 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63bd85c3_8a0d_44a1_99cd_0e1e727f2c90.slice/crio-2776a08025dfbf1170d10c7373937aec7efcd1909d97d1f66d31daebfd88da90 WatchSource:0}: Error finding container 2776a08025dfbf1170d10c7373937aec7efcd1909d97d1f66d31daebfd88da90: Status 404 returned error can't find the container with id 2776a08025dfbf1170d10c7373937aec7efcd1909d97d1f66d31daebfd88da90 Jun 02 21:51:22.477871 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:22.477835 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" event={"ID":"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90","Type":"ContainerStarted","Data":"2776a08025dfbf1170d10c7373937aec7efcd1909d97d1f66d31daebfd88da90"} Jun 02 21:51:26.493723 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:26.493681 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" event={"ID":"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90","Type":"ContainerStarted","Data":"05fe1b6905218f7a8a1ab601c75e981e6da1a72812b6ba3799c0e13167f2ccc3"} Jun 02 21:51:27.498524 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:27.498427 2558 generic.go:358] "Generic (PLEG): container finished" podID="63bd85c3-8a0d-44a1-99cd-0e1e727f2c90" containerID="05fe1b6905218f7a8a1ab601c75e981e6da1a72812b6ba3799c0e13167f2ccc3" exitCode=0 Jun 02 21:51:27.498524 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:27.498509 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" event={"ID":"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90","Type":"ContainerDied","Data":"05fe1b6905218f7a8a1ab601c75e981e6da1a72812b6ba3799c0e13167f2ccc3"} Jun 02 21:51:29.508710 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:29.508672 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" event={"ID":"63bd85c3-8a0d-44a1-99cd-0e1e727f2c90","Type":"ContainerStarted","Data":"587645d5af9394e34366a69aad202174bfc7689b5db0dc4757b7359345fd5238"} Jun 02 21:51:29.527960 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:29.527897 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" podStartSLOduration=2.036549695 podStartE2EDuration="8.527880515s" podCreationTimestamp="2026-06-02 21:51:21 +0000 UTC" firstStartedPulling="2026-06-02 21:51:22.147569841 +0000 UTC m=+709.948930800" lastFinishedPulling="2026-06-02 21:51:28.638900668 +0000 UTC m=+716.440261620" observedRunningTime="2026-06-02 21:51:29.525908293 +0000 UTC m=+717.327269270" watchObservedRunningTime="2026-06-02 21:51:29.527880515 +0000 UTC m=+717.329241490" Jun 02 21:51:32.005563 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:32.005513 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:32.005563 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:32.005563 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:32.018596 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:32.018564 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:32.531489 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:32.531460 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r" Jun 02 21:51:47.315546 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.315509 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc"] Jun 02 21:51:47.335492 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.335451 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc"] Jun 02 21:51:47.335656 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.335608 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.338094 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.338068 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 02 21:51:47.440943 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.440900 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b71a8a53-d73d-4b52-b594-34a80e5ae15d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.440943 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.440940 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.441223 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.441007 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.441223 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.441054 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.441223 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.441083 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.441223 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.441151 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5zpk\" (UniqueName: \"kubernetes.io/projected/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kube-api-access-q5zpk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.541949 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.541900 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.541949 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.541952 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.541980 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5zpk\" (UniqueName: \"kubernetes.io/projected/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kube-api-access-q5zpk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542050 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b71a8a53-d73d-4b52-b594-34a80e5ae15d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542068 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542216 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542105 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542486 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542462 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542570 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542491 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.542570 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.542526 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.544362 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.544338 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b71a8a53-d73d-4b52-b594-34a80e5ae15d-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.544598 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.544580 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b71a8a53-d73d-4b52-b594-34a80e5ae15d-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.550171 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.550147 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5zpk\" (UniqueName: \"kubernetes.io/projected/b71a8a53-d73d-4b52-b594-34a80e5ae15d-kube-api-access-q5zpk\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-zxntc\" (UID: \"b71a8a53-d73d-4b52-b594-34a80e5ae15d\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.648184 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.648148 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:47.784136 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.784109 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc"] Jun 02 21:51:47.786604 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:51:47.786574 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71a8a53_d73d_4b52_b594_34a80e5ae15d.slice/crio-7157c8f720d8446c38043a20187dfd921e40e42141dd233db37a41f16b9f298f WatchSource:0}: Error finding container 7157c8f720d8446c38043a20187dfd921e40e42141dd233db37a41f16b9f298f: Status 404 returned error can't find the container with id 7157c8f720d8446c38043a20187dfd921e40e42141dd233db37a41f16b9f298f Jun 02 21:51:47.835844 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:47.835812 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:51:48.579517 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:48.579477 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" event={"ID":"b71a8a53-d73d-4b52-b594-34a80e5ae15d","Type":"ContainerStarted","Data":"3d0c06e197b77462f75aee92798f721de8a96a18b3bf393a24133fca014f8dc8"} Jun 02 21:51:48.579517 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:48.579524 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" event={"ID":"b71a8a53-d73d-4b52-b594-34a80e5ae15d","Type":"ContainerStarted","Data":"7157c8f720d8446c38043a20187dfd921e40e42141dd233db37a41f16b9f298f"} Jun 02 21:51:50.588418 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:50.588386 2558 generic.go:358] "Generic (PLEG): container finished" podID="b71a8a53-d73d-4b52-b594-34a80e5ae15d" containerID="3d0c06e197b77462f75aee92798f721de8a96a18b3bf393a24133fca014f8dc8" exitCode=0 Jun 02 21:51:50.588805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:50.588454 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" event={"ID":"b71a8a53-d73d-4b52-b594-34a80e5ae15d","Type":"ContainerDied","Data":"3d0c06e197b77462f75aee92798f721de8a96a18b3bf393a24133fca014f8dc8"} Jun 02 21:51:56.613585 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:56.613545 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" event={"ID":"b71a8a53-d73d-4b52-b594-34a80e5ae15d","Type":"ContainerStarted","Data":"4775d3ea0b7e431e3b7826eb7c62fe6612332abe101e353ca73acbac88ad0148"} Jun 02 21:51:56.632702 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:56.632645 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" podStartSLOduration=4.154443013 podStartE2EDuration="9.632627384s" podCreationTimestamp="2026-06-02 21:51:47 +0000 UTC" firstStartedPulling="2026-06-02 21:51:50.589110604 +0000 UTC m=+738.390471558" lastFinishedPulling="2026-06-02 21:51:56.067294961 +0000 UTC m=+743.868655929" observedRunningTime="2026-06-02 21:51:56.630296866 +0000 UTC m=+744.431657843" watchObservedRunningTime="2026-06-02 21:51:56.632627384 +0000 UTC m=+744.433988358" Jun 02 21:51:57.649279 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:57.649225 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:57.649715 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:57.649550 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:57.662377 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:57.662339 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:58.632079 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:58.632050 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-zxntc" Jun 02 21:51:59.703197 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.703154 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z"] Jun 02 21:51:59.707462 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.707435 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.709901 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.709879 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 02 21:51:59.715220 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.715190 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z"] Jun 02 21:51:59.738185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738147 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40c50fe-0c55-4bcf-922a-b32c20e5706a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.738352 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738228 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.738352 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738254 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.738352 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738278 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.738352 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738326 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.738481 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.738390 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49vrz\" (UniqueName: \"kubernetes.io/projected/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kube-api-access-49vrz\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.838856 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.838818 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-49vrz\" (UniqueName: \"kubernetes.io/projected/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kube-api-access-49vrz\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.838856 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.838863 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40c50fe-0c55-4bcf-922a-b32c20e5706a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.838935 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.838960 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.839024 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.839068 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.839388 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.839426 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-home\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.839650 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.839507 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.841442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.841408 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a40c50fe-0c55-4bcf-922a-b32c20e5706a-dshm\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.841955 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.841931 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a40c50fe-0c55-4bcf-922a-b32c20e5706a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:51:59.847013 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:51:59.846949 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49vrz\" (UniqueName: \"kubernetes.io/projected/a40c50fe-0c55-4bcf-922a-b32c20e5706a-kube-api-access-49vrz\") pod \"e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z\" (UID: \"a40c50fe-0c55-4bcf-922a-b32c20e5706a\") " pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:52:00.020659 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:00.020559 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" Jun 02 21:52:00.156761 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:00.156683 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z"] Jun 02 21:52:00.159675 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:52:00.159640 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda40c50fe_0c55_4bcf_922a_b32c20e5706a.slice/crio-3d8f0cf123f4c6b7b05c4a96e57533b97e7a3abf70d53f5734b1d756b596cea3 WatchSource:0}: Error finding container 3d8f0cf123f4c6b7b05c4a96e57533b97e7a3abf70d53f5734b1d756b596cea3: Status 404 returned error can't find the container with id 3d8f0cf123f4c6b7b05c4a96e57533b97e7a3abf70d53f5734b1d756b596cea3 Jun 02 21:52:00.523855 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:00.523812 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:52:00.629713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:00.629674 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" event={"ID":"a40c50fe-0c55-4bcf-922a-b32c20e5706a","Type":"ContainerStarted","Data":"28882a328c2b2d7cd02d27ad9d3fb0cd29816e2c6c5220cd77d0d60e9c4ce70e"} Jun 02 21:52:00.629713 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:00.629719 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z" event={"ID":"a40c50fe-0c55-4bcf-922a-b32c20e5706a","Type":"ContainerStarted","Data":"3d8f0cf123f4c6b7b05c4a96e57533b97e7a3abf70d53f5734b1d756b596cea3"} Jun 02 21:52:16.510107 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.510065 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v"] Jun 02 21:52:16.516102 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.516074 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.518508 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.518481 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 02 21:52:16.522566 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.522535 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v"] Jun 02 21:52:16.596167 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596127 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn72d\" (UniqueName: \"kubernetes.io/projected/e986878b-3895-4e06-9c40-b6107ea0428c-kube-api-access-nn72d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.596376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596178 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.596376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596211 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.596376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596280 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e986878b-3895-4e06-9c40-b6107ea0428c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.596376 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596322 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.596556 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.596395 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697322 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697281 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697333 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nn72d\" (UniqueName: \"kubernetes.io/projected/e986878b-3895-4e06-9c40-b6107ea0428c-kube-api-access-nn72d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697361 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697389 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697419 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e986878b-3895-4e06-9c40-b6107ea0428c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697533 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697441 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697768 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697805 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697792 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.697926 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.697903 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.699766 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.699740 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e986878b-3895-4e06-9c40-b6107ea0428c-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.700122 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.700104 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e986878b-3895-4e06-9c40-b6107ea0428c-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.705936 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.705908 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn72d\" (UniqueName: \"kubernetes.io/projected/e986878b-3895-4e06-9c40-b6107ea0428c-kube-api-access-nn72d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v\" (UID: \"e986878b-3895-4e06-9c40-b6107ea0428c\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.831114 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.830984 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:16.973648 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:16.973616 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v"] Jun 02 21:52:16.976119 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:52:16.976087 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode986878b_3895_4e06_9c40_b6107ea0428c.slice/crio-bd7110295493f047d59a3ee80c7980077ece5f90e48f2845217d96775d7624e7 WatchSource:0}: Error finding container bd7110295493f047d59a3ee80c7980077ece5f90e48f2845217d96775d7624e7: Status 404 returned error can't find the container with id bd7110295493f047d59a3ee80c7980077ece5f90e48f2845217d96775d7624e7 Jun 02 21:52:17.426958 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:17.426918 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:52:17.691663 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:17.691575 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" event={"ID":"e986878b-3895-4e06-9c40-b6107ea0428c","Type":"ContainerStarted","Data":"53aba7e96a7cc8da280df85557de9b1b57a343f05098f915f16e9c7e314619a4"} Jun 02 21:52:17.691663 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:17.691611 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" event={"ID":"e986878b-3895-4e06-9c40-b6107ea0428c","Type":"ContainerStarted","Data":"bd7110295493f047d59a3ee80c7980077ece5f90e48f2845217d96775d7624e7"} Jun 02 21:52:18.697844 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:18.697805 2558 generic.go:358] "Generic (PLEG): container finished" podID="e986878b-3895-4e06-9c40-b6107ea0428c" containerID="53aba7e96a7cc8da280df85557de9b1b57a343f05098f915f16e9c7e314619a4" exitCode=0 Jun 02 21:52:18.698309 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:18.697894 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" event={"ID":"e986878b-3895-4e06-9c40-b6107ea0428c","Type":"ContainerDied","Data":"53aba7e96a7cc8da280df85557de9b1b57a343f05098f915f16e9c7e314619a4"} Jun 02 21:52:19.702829 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:19.702792 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" event={"ID":"e986878b-3895-4e06-9c40-b6107ea0428c","Type":"ContainerStarted","Data":"9365a6e524556de9c1ef57fdc1265e67c3b2b572bd7d428b85cad0cad05c68f5"} Jun 02 21:52:19.722515 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:19.722457 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" podStartSLOduration=3.50866903 podStartE2EDuration="3.722435082s" podCreationTimestamp="2026-06-02 21:52:16 +0000 UTC" firstStartedPulling="2026-06-02 21:52:18.698395466 +0000 UTC m=+766.499756418" lastFinishedPulling="2026-06-02 21:52:18.912161503 +0000 UTC m=+766.713522470" observedRunningTime="2026-06-02 21:52:19.719822275 +0000 UTC m=+767.521183250" watchObservedRunningTime="2026-06-02 21:52:19.722435082 +0000 UTC m=+767.523796057" Jun 02 21:52:26.831965 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:26.831923 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:26.831965 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:26.831968 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:26.844813 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:26.844779 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:27.749686 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:27.749647 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v" Jun 02 21:52:29.494788 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.494751 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8"] Jun 02 21:52:29.498951 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.498920 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.501299 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.501275 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 02 21:52:29.509492 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.509460 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8"] Jun 02 21:52:29.619096 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619055 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.619293 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619104 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78429c5c-1d89-46a6-9b68-af99a8bcb06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.619293 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619170 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcdkq\" (UniqueName: \"kubernetes.io/projected/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kube-api-access-xcdkq\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.619386 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619295 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.619386 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619332 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.619487 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.619414 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720080 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720034 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720286 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720089 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78429c5c-1d89-46a6-9b68-af99a8bcb06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720286 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720124 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xcdkq\" (UniqueName: \"kubernetes.io/projected/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kube-api-access-xcdkq\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720286 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720211 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720286 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720248 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720298 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720508 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720571 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720567 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.720735 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.720630 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.722424 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.722401 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78429c5c-1d89-46a6-9b68-af99a8bcb06e-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.722703 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.722687 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78429c5c-1d89-46a6-9b68-af99a8bcb06e-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.727213 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.727190 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcdkq\" (UniqueName: \"kubernetes.io/projected/78429c5c-1d89-46a6-9b68-af99a8bcb06e-kube-api-access-xcdkq\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8\" (UID: \"78429c5c-1d89-46a6-9b68-af99a8bcb06e\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.811282 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.811179 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:29.950758 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:29.950720 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8"] Jun 02 21:52:29.954116 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:52:29.954086 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78429c5c_1d89_46a6_9b68_af99a8bcb06e.slice/crio-67ec3ac80abc2e19a8278e114842c04bb8257084ac189b9a65d530ff39882405 WatchSource:0}: Error finding container 67ec3ac80abc2e19a8278e114842c04bb8257084ac189b9a65d530ff39882405: Status 404 returned error can't find the container with id 67ec3ac80abc2e19a8278e114842c04bb8257084ac189b9a65d530ff39882405 Jun 02 21:52:30.751500 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:30.751466 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" event={"ID":"78429c5c-1d89-46a6-9b68-af99a8bcb06e","Type":"ContainerStarted","Data":"9817f33ee9ed70ec1e9b37eeb45efe4f90b67f777f2afca6e2d4516e4257fc51"} Jun 02 21:52:30.751500 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:30.751503 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" event={"ID":"78429c5c-1d89-46a6-9b68-af99a8bcb06e","Type":"ContainerStarted","Data":"67ec3ac80abc2e19a8278e114842c04bb8257084ac189b9a65d530ff39882405"} Jun 02 21:52:30.920270 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:30.920226 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:52:31.756238 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:31.756201 2558 generic.go:358] "Generic (PLEG): container finished" podID="78429c5c-1d89-46a6-9b68-af99a8bcb06e" containerID="9817f33ee9ed70ec1e9b37eeb45efe4f90b67f777f2afca6e2d4516e4257fc51" exitCode=0 Jun 02 21:52:31.756621 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:31.756286 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" event={"ID":"78429c5c-1d89-46a6-9b68-af99a8bcb06e","Type":"ContainerDied","Data":"9817f33ee9ed70ec1e9b37eeb45efe4f90b67f777f2afca6e2d4516e4257fc51"} Jun 02 21:52:32.761929 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:32.761888 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" event={"ID":"78429c5c-1d89-46a6-9b68-af99a8bcb06e","Type":"ContainerStarted","Data":"b7e36d99cfe3df082f8b510b80378e1af82065e2c3166064179fbfcb1f3dcf77"} Jun 02 21:52:32.781903 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:32.781851 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" podStartSLOduration=3.590853225 podStartE2EDuration="3.781837332s" podCreationTimestamp="2026-06-02 21:52:29 +0000 UTC" firstStartedPulling="2026-06-02 21:52:31.756807887 +0000 UTC m=+779.558168839" lastFinishedPulling="2026-06-02 21:52:31.94779199 +0000 UTC m=+779.749152946" observedRunningTime="2026-06-02 21:52:32.780853766 +0000 UTC m=+780.582214744" watchObservedRunningTime="2026-06-02 21:52:32.781837332 +0000 UTC m=+780.583198307" Jun 02 21:52:35.321464 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:35.321425 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-fcfmz"] Jun 02 21:52:39.811728 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:39.811682 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:39.811728 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:39.811737 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:39.824701 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:39.824674 2558 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:52:40.803566 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:52:40.803539 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8" Jun 02 21:54:32.764597 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:32.764567 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:54:32.766724 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:32.766699 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:54:48.633776 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:48.633749 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-8c7bb9f47-zmrwq_43794281-b96c-4866-acbb-e7af09d6c096/nginx/0.log" Jun 02 21:54:48.739241 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:48.739207 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-h5hnr_8a49494c-6ab7-4b0c-bccf-0a9def83c472/manager/0.log" Jun 02 21:54:48.853758 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:48.853726 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-tw9l2_29d90541-7c1c-4b27-9f5e-17b028ef3f43/manager/0.log" Jun 02 21:54:48.961583 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:48.961553 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-85d8c9f8f6-6nt7d_7ab29c97-52a0-4347-abad-6b4f7e7027f4/maas-api/0.log" Jun 02 21:54:49.289033 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:49.288930 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-d76t4_a243b531-9ec9-4e51-93f7-45ecc70e24b8/manager/0.log" Jun 02 21:54:49.507820 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:49.507794 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d44dd895c-ntq4g_61f01710-6877-4245-ad32-b5b4565054c1/copy-manifests/0.log" Jun 02 21:54:49.520572 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:49.520543 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d44dd895c-ntq4g_61f01710-6877-4245-ad32-b5b4565054c1/manager/0.log" Jun 02 21:54:49.735397 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:49.735370 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9r9wl_f199ed9f-72ea-412c-8fa9-446ac6ecd810/postgres/0.log" Jun 02 21:54:51.147768 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:51.147743 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-ltmlt_eab6e1bd-38cb-4232-b18f-ccf04194dcc9/manager/0.log" Jun 02 21:54:51.583442 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:51.583363 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-fcfmz_9d288fe2-7a8b-4e3d-a7dc-dbdae5286683/limitador/0.log" Jun 02 21:54:52.138328 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:52.138300 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-g88ts_d9b891ec-69f2-4a6c-a1d0-68213d829018/discovery/0.log" Jun 02 21:54:52.344760 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:52.344731 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-567f8cc9bf-xqt6p_bad53864-7d52-4c10-9f47-26642ed53d69/kube-auth-proxy/0.log" Jun 02 21:54:52.994430 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:52.994402 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r_63bd85c3-8a0d-44a1-99cd-0e1e727f2c90/storage-initializer/0.log" Jun 02 21:54:53.000580 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.000560 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-8595bdc8d4-7bz9r_63bd85c3-8a0d-44a1-99cd-0e1e727f2c90/main/0.log" Jun 02 21:54:53.106366 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.106338 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-zxntc_b71a8a53-d73d-4b52-b594-34a80e5ae15d/storage-initializer/0.log" Jun 02 21:54:53.113202 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.113178 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-zxntc_b71a8a53-d73d-4b52-b594-34a80e5ae15d/main/0.log" Jun 02 21:54:53.216398 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.216355 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84f46c7fb4-l6h7z_a40c50fe-0c55-4bcf-922a-b32c20e5706a/storage-initializer/0.log" Jun 02 21:54:53.326142 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.326057 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v_e986878b-3895-4e06-9c40-b6107ea0428c/storage-initializer/0.log" Jun 02 21:54:53.332641 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.332621 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9t4b2v_e986878b-3895-4e06-9c40-b6107ea0428c/main/0.log" Jun 02 21:54:53.439655 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.439629 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8_78429c5c-1d89-46a6-9b68-af99a8bcb06e/storage-initializer/0.log" Jun 02 21:54:53.446240 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:54:53.446216 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-dlrn8_78429c5c-1d89-46a6-9b68-af99a8bcb06e/main/0.log" Jun 02 21:55:00.311522 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:00.311490 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tpm87_e6fcc949-f931-481e-8bce-c896aaaa5189/global-pull-secret-syncer/0.log" Jun 02 21:55:00.386541 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:00.386514 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-srmjb_d35d06ad-ad18-49ea-8889-caafae090e7b/konnectivity-agent/0.log" Jun 02 21:55:00.476040 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:00.476013 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-61.ec2.internal_23c71c5268172c8479e10de3f191f37f/haproxy/0.log" Jun 02 21:55:04.426530 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:04.426496 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-ltmlt_eab6e1bd-38cb-4232-b18f-ccf04194dcc9/manager/0.log" Jun 02 21:55:04.561534 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:04.561506 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-fcfmz_9d288fe2-7a8b-4e3d-a7dc-dbdae5286683/limitador/0.log" Jun 02 21:55:06.186635 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:06.186602 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4g892_bcbf5c21-8561-4d56-a079-801a2e3b2eb5/node-exporter/0.log" Jun 02 21:55:06.211946 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:06.211916 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4g892_bcbf5c21-8561-4d56-a079-801a2e3b2eb5/kube-rbac-proxy/0.log" Jun 02 21:55:06.236716 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:06.236692 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4g892_bcbf5c21-8561-4d56-a079-801a2e3b2eb5/init-textfile/0.log" Jun 02 21:55:08.243591 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:08.243554 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-5nlzv_775cb3c0-90b1-4201-b45d-9255bb856130/networking-console-plugin/0.log" Jun 02 21:55:08.818138 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:08.818106 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/1.log" Jun 02 21:55:08.823879 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:08.823856 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-86d7c6cc77-88rls_658df708-29d1-44e2-83d5-33e5316847f7/console-operator/2.log" Jun 02 21:55:09.852555 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.852517 2558 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv"] Jun 02 21:55:09.855884 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.855868 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:09.858538 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.858516 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zpld4\"/\"kube-root-ca.crt\"" Jun 02 21:55:09.858681 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.858569 2558 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zpld4\"/\"openshift-service-ca.crt\"" Jun 02 21:55:09.859661 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.859645 2558 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zpld4\"/\"default-dockercfg-fzxl8\"" Jun 02 21:55:09.866260 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.866233 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv"] Jun 02 21:55:09.879236 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.879199 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-7598fd5bd-g5rnm_7f0e74e3-e3bf-4668-872f-d3b519382477/volume-data-source-validator/0.log" Jun 02 21:55:09.991936 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.991906 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-proc\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:09.992161 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.991960 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-sys\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:09.992233 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.992200 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-lib-modules\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:09.992292 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.992232 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-podres\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:09.992402 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:09.992384 2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqrl\" (UniqueName: \"kubernetes.io/projected/f61d20a0-427f-468b-89fd-032e35f76458-kube-api-access-4fqrl\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.093881 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.093841 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-sys\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.093881 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.093892 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-lib-modules\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.093983 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-podres\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094010 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-sys\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094081 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqrl\" (UniqueName: \"kubernetes.io/projected/f61d20a0-427f-468b-89fd-032e35f76458-kube-api-access-4fqrl\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094121 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-lib-modules\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094129 2558 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-proc\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094135 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-podres\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.094185 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.094173 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f61d20a0-427f-468b-89fd-032e35f76458-proc\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.102048 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.102013 2558 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqrl\" (UniqueName: \"kubernetes.io/projected/f61d20a0-427f-468b-89fd-032e35f76458-kube-api-access-4fqrl\") pod \"perf-node-gather-daemonset-6b8hv\" (UID: \"f61d20a0-427f-468b-89fd-032e35f76458\") " pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.167282 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.167226 2558 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:10.304959 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.304920 2558 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv"] Jun 02 21:55:10.309005 ip-10-0-143-61 kubenswrapper[2558]: W0602 21:55:10.308960 2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf61d20a0_427f_468b_89fd_032e35f76458.slice/crio-a0682f161dd9f3d4836c2f2b49847103b7668da15999bac1426aca94a7b4a2f2 WatchSource:0}: Error finding container a0682f161dd9f3d4836c2f2b49847103b7668da15999bac1426aca94a7b4a2f2: Status 404 returned error can't find the container with id a0682f161dd9f3d4836c2f2b49847103b7668da15999bac1426aca94a7b4a2f2 Jun 02 21:55:10.349615 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.349579 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" event={"ID":"f61d20a0-427f-468b-89fd-032e35f76458","Type":"ContainerStarted","Data":"a0682f161dd9f3d4836c2f2b49847103b7668da15999bac1426aca94a7b4a2f2"} Jun 02 21:55:10.870577 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.870492 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhxs5_64c54cf9-214c-4a6e-bddb-cfc67303da34/dns/0.log" Jun 02 21:55:10.905052 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.905020 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qhxs5_64c54cf9-214c-4a6e-bddb-cfc67303da34/kube-rbac-proxy/0.log" Jun 02 21:55:10.945885 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:10.945853 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mqkj_037ea9d6-87f1-492a-9026-58e056b65527/dns-node-resolver/0.log" Jun 02 21:55:11.354131 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:11.354092 2558 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" event={"ID":"f61d20a0-427f-468b-89fd-032e35f76458","Type":"ContainerStarted","Data":"537d6a24b697382c79a995a8e259ec6b6d4e1d84b9889908070167dfa2085589"} Jun 02 21:55:11.354331 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:11.354219 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:11.369579 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:11.368529 2558 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" podStartSLOduration=2.367926057 podStartE2EDuration="2.367926057s" podCreationTimestamp="2026-06-02 21:55:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:55:11.367047425 +0000 UTC m=+939.168408401" watchObservedRunningTime="2026-06-02 21:55:11.367926057 +0000 UTC m=+939.169287031" Jun 02 21:55:11.526629 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:11.526594 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-npgt5_c7859d70-6355-4231-89d0-5b0fa8e8a79c/node-ca/0.log" Jun 02 21:55:12.488643 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:12.488607 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-g88ts_d9b891ec-69f2-4a6c-a1d0-68213d829018/discovery/0.log" Jun 02 21:55:12.534378 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:12.534348 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-567f8cc9bf-xqt6p_bad53864-7d52-4c10-9f47-26642ed53d69/kube-auth-proxy/0.log" Jun 02 21:55:13.223939 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:13.223906 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4pbgs_d02a7b59-6742-4b76-8286-e8b53f9df26d/serve-healthcheck-canary/0.log" Jun 02 21:55:13.850982 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:13.850932 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5lr4_15732505-5698-4bdf-8983-7c4f10a6394d/kube-rbac-proxy/0.log" Jun 02 21:55:13.873363 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:13.873333 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5lr4_15732505-5698-4bdf-8983-7c4f10a6394d/exporter/0.log" Jun 02 21:55:13.894578 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:13.894554 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n5lr4_15732505-5698-4bdf-8983-7c4f10a6394d/extractor/0.log" Jun 02 21:55:15.805833 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:15.805797 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-8c7bb9f47-zmrwq_43794281-b96c-4866-acbb-e7af09d6c096/nginx/0.log" Jun 02 21:55:15.832225 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:15.832191 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-7c99ffcdcd-h5hnr_8a49494c-6ab7-4b0c-bccf-0a9def83c472/manager/0.log" Jun 02 21:55:15.867776 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:15.867746 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-c6b9847d7-tw9l2_29d90541-7c1c-4b27-9f5e-17b028ef3f43/manager/0.log" Jun 02 21:55:15.886748 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:15.886717 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-85d8c9f8f6-6nt7d_7ab29c97-52a0-4347-abad-6b4f7e7027f4/maas-api/0.log" Jun 02 21:55:15.976699 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:15.976664 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-d76t4_a243b531-9ec9-4e51-93f7-45ecc70e24b8/manager/0.log" Jun 02 21:55:16.080123 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:16.080041 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d44dd895c-ntq4g_61f01710-6877-4245-ad32-b5b4565054c1/manager/0.log" Jun 02 21:55:16.097208 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:16.097180 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d44dd895c-ntq4g_61f01710-6877-4245-ad32-b5b4565054c1/copy-manifests/0.log" Jun 02 21:55:16.163610 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:16.163569 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-9r9wl_f199ed9f-72ea-412c-8fa9-446ac6ecd810/postgres/0.log" Jun 02 21:55:17.369072 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:17.369042 2558 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zpld4/perf-node-gather-daemonset-6b8hv" Jun 02 21:55:22.784293 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:22.784224 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2swqs_8796e9bb-846a-48ef-88be-45ac60ffc2fe/kube-storage-version-migrator-operator/1.log" Jun 02 21:55:22.784950 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:22.784920 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-96ddc787c-2swqs_8796e9bb-846a-48ef-88be-45ac60ffc2fe/kube-storage-version-migrator-operator/0.log" Jun 02 21:55:23.789295 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.789204 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/kube-multus-additional-cni-plugins/0.log" Jun 02 21:55:23.809703 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.809673 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/egress-router-binary-copy/0.log" Jun 02 21:55:23.831466 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.831438 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/cni-plugins/0.log" Jun 02 21:55:23.851832 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.851806 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/bond-cni-plugin/0.log" Jun 02 21:55:23.875659 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.875625 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/routeoverride-cni/0.log" Jun 02 21:55:23.896809 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.896778 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/whereabouts-cni-bincopy/0.log" Jun 02 21:55:23.916495 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:23.916464 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2k9mn_744ab60f-c16e-4f34-8d7a-879cde111ece/whereabouts-cni/0.log" Jun 02 21:55:24.269835 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:24.269801 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tjhms_0f93e131-2668-45a1-ba9b-d207128c0de1/kube-multus/0.log" Jun 02 21:55:24.288910 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:24.288877 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s6gxd_afdad777-7975-4eef-bc20-ef9878174b53/network-metrics-daemon/0.log" Jun 02 21:55:24.308068 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:24.308009 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-s6gxd_afdad777-7975-4eef-bc20-ef9878174b53/kube-rbac-proxy/0.log" Jun 02 21:55:25.460733 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.460704 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/ovn-controller/0.log" Jun 02 21:55:25.490199 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.490161 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/ovn-acl-logging/0.log" Jun 02 21:55:25.508778 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.508750 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/kube-rbac-proxy-node/0.log" Jun 02 21:55:25.531807 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.531772 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 21:55:25.549568 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.549544 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/northd/0.log" Jun 02 21:55:25.570076 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.570029 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/nbdb/0.log" Jun 02 21:55:25.590162 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.590129 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/sbdb/0.log" Jun 02 21:55:25.705083 ip-10-0-143-61 kubenswrapper[2558]: I0602 21:55:25.705053 2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rwwhv_bc2174cc-1c22-45c9-987f-5cd42bbfa9a5/ovnkube-controller/0.log"