May 28 20:51:52.251736 ip-10-0-136-77 systemd[1]: Starting Kubernetes Kubelet... May 28 20:51:52.756102 ip-10-0-136-77 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:51:52.756102 ip-10-0-136-77 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 28 20:51:52.756102 ip-10-0-136-77 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:51:52.756102 ip-10-0-136-77 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 28 20:51:52.756102 ip-10-0-136-77 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 28 20:51:52.760210 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.760071 2570 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766523 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766542 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766548 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766551 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766555 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:51:52.766549 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766558 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766561 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766564 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766567 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766570 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766573 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766576 2570 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766578 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766581 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766583 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766586 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766588 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766591 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766594 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766596 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766599 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766601 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766604 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766606 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766609 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:51:52.766781 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766612 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766614 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766617 2570 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766619 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766622 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766624 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766628 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766630 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766633 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766637 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766642 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766644 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766648 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766653 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766658 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766661 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766665 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766668 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766671 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:51:52.767261 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766674 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766676 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766680 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766682 2570 feature_gate.go:328] unrecognized feature gate: Example May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766685 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766687 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766690 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766693 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766696 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766698 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766701 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766704 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766706 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766709 2570 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766712 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766716 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766719 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766721 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766724 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766726 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766729 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:51:52.767749 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766731 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766734 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766736 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766739 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766742 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766744 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766748 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766751 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766754 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766756 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766759 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766761 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766764 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766766 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766769 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766772 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766774 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766777 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766779 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:51:52.768255 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766782 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.766785 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.767994 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768000 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768004 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768007 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768010 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768013 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768015 2570 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768018 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768021 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768023 2570 feature_gate.go:328] unrecognized feature gate: Example May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768026 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768029 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768031 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768034 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768036 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768039 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768041 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768044 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:51:52.768756 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768047 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768050 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768052 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768055 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768057 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768060 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768063 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768066 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768068 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768071 2570 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768073 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768076 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768078 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768081 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768083 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768086 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768088 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768091 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768093 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768096 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:51:52.769238 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768098 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768101 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768103 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768105 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768108 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768110 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768113 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768115 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768118 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768120 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768123 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768127 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768129 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768132 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768134 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768137 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768139 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768142 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768144 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:51:52.769761 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768147 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768151 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768154 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768157 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768160 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768162 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768166 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768168 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768171 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768173 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768176 2570 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768178 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768181 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768183 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768186 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768188 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768191 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768193 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768196 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:51:52.770231 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768198 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768201 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768205 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768209 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768212 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768216 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768219 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768222 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768224 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.768227 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769097 2570 flags.go:64] FLAG: --address="0.0.0.0" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769107 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769114 2570 flags.go:64] FLAG: --anonymous-auth="true" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769123 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769129 2570 flags.go:64] FLAG: --authentication-token-webhook="false" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769133 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769137 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769141 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769145 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769148 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769151 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 28 20:51:52.770710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769155 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769158 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769161 2570 flags.go:64] FLAG: --cgroup-root="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769163 2570 flags.go:64] FLAG: --cgroups-per-qos="true" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769167 2570 flags.go:64] FLAG: --client-ca-file="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769170 2570 flags.go:64] FLAG: --cloud-config="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769173 2570 flags.go:64] FLAG: --cloud-provider="external" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769176 2570 flags.go:64] FLAG: --cluster-dns="[]" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769180 2570 flags.go:64] FLAG: --cluster-domain="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769183 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769186 2570 flags.go:64] FLAG: --config-dir="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769189 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769193 2570 flags.go:64] FLAG: --container-log-max-files="5" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769196 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769199 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769202 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769206 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769209 2570 flags.go:64] FLAG: --contention-profiling="false" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769212 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769215 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769219 2570 flags.go:64] FLAG: --cpu-manager-policy="none" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769222 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769227 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769230 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769233 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" May 28 20:51:52.771221 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769236 2570 flags.go:64] FLAG: --enable-load-reader="false" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769240 2570 flags.go:64] FLAG: --enable-server="true" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769243 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769247 2570 flags.go:64] FLAG: --event-burst="100" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769251 2570 flags.go:64] FLAG: --event-qps="50" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769254 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769257 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769260 2570 flags.go:64] FLAG: --eviction-hard="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769264 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769267 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769270 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769273 2570 flags.go:64] FLAG: --eviction-soft="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769276 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769279 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769282 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769285 2570 flags.go:64] FLAG: --experimental-mounter-path="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769288 2570 flags.go:64] FLAG: --fail-cgroupv1="false" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769291 2570 flags.go:64] FLAG: --fail-swap-on="true" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769294 2570 flags.go:64] FLAG: --feature-gates="" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769298 2570 flags.go:64] FLAG: --file-check-frequency="20s" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769301 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769304 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769307 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769310 2570 flags.go:64] FLAG: --healthz-port="10248" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769313 2570 flags.go:64] FLAG: --help="false" May 28 20:51:52.771841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769316 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-136-77.ec2.internal" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769320 2570 flags.go:64] FLAG: --housekeeping-interval="10s" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769323 2570 flags.go:64] FLAG: --http-check-frequency="20s" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769326 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769330 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769333 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769337 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769340 2570 flags.go:64] FLAG: --image-service-endpoint="" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769342 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769346 2570 flags.go:64] FLAG: --kube-api-burst="100" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769349 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769352 2570 flags.go:64] FLAG: --kube-api-qps="50" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769355 2570 flags.go:64] FLAG: --kube-reserved="" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769358 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769361 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769364 2570 flags.go:64] FLAG: --kubelet-cgroups="" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769367 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769370 2570 flags.go:64] FLAG: --lock-file="" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769373 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769376 2570 flags.go:64] FLAG: --log-flush-frequency="5s" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769378 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769384 2570 flags.go:64] FLAG: --log-json-split-stream="false" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769387 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769390 2570 flags.go:64] FLAG: --log-text-split-stream="false" May 28 20:51:52.772434 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769393 2570 flags.go:64] FLAG: --logging-format="text" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769396 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769400 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769403 2570 flags.go:64] FLAG: --manifest-url="" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769406 2570 flags.go:64] FLAG: --manifest-url-header="" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769410 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769413 2570 flags.go:64] FLAG: --max-open-files="1000000" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769417 2570 flags.go:64] FLAG: --max-pods="110" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769420 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769423 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769427 2570 flags.go:64] FLAG: --memory-manager-policy="None" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769430 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769433 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769436 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769440 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769447 2570 flags.go:64] FLAG: --node-status-max-images="50" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769451 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769454 2570 flags.go:64] FLAG: --oom-score-adj="-999" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769457 2570 flags.go:64] FLAG: --pod-cidr="" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769461 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769465 2570 flags.go:64] FLAG: --pod-manifest-path="" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769481 2570 flags.go:64] FLAG: --pod-max-pids="-1" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769484 2570 flags.go:64] FLAG: --pods-per-core="0" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769487 2570 flags.go:64] FLAG: --port="10250" May 28 20:51:52.773025 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769491 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769494 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00da252adc335ba5c" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769497 2570 flags.go:64] FLAG: --qos-reserved="" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769500 2570 flags.go:64] FLAG: --read-only-port="10255" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769503 2570 flags.go:64] FLAG: --register-node="true" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769506 2570 flags.go:64] FLAG: --register-schedulable="true" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769509 2570 flags.go:64] FLAG: --register-with-taints="" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769518 2570 flags.go:64] FLAG: --registry-burst="10" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769521 2570 flags.go:64] FLAG: --registry-qps="5" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769524 2570 flags.go:64] FLAG: --reserved-cpus="" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769527 2570 flags.go:64] FLAG: --reserved-memory="" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769531 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769534 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769537 2570 flags.go:64] FLAG: --rotate-certificates="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769539 2570 flags.go:64] FLAG: --rotate-server-certificates="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769542 2570 flags.go:64] FLAG: --runonce="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769545 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769548 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769551 2570 flags.go:64] FLAG: --seccomp-default="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769554 2570 flags.go:64] FLAG: --serialize-image-pulls="true" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769557 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769560 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769563 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769566 2570 flags.go:64] FLAG: --storage-driver-password="root" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769569 2570 flags.go:64] FLAG: --storage-driver-secure="false" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769572 2570 flags.go:64] FLAG: --storage-driver-table="stats" May 28 20:51:52.773644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769575 2570 flags.go:64] FLAG: --storage-driver-user="root" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769579 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769582 2570 flags.go:64] FLAG: --sync-frequency="1m0s" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769584 2570 flags.go:64] FLAG: --system-cgroups="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769587 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769593 2570 flags.go:64] FLAG: --system-reserved-cgroup="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769596 2570 flags.go:64] FLAG: --tls-cert-file="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769599 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769603 2570 flags.go:64] FLAG: --tls-min-version="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769606 2570 flags.go:64] FLAG: --tls-private-key-file="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769611 2570 flags.go:64] FLAG: --topology-manager-policy="none" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769614 2570 flags.go:64] FLAG: --topology-manager-policy-options="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769617 2570 flags.go:64] FLAG: --topology-manager-scope="container" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769622 2570 flags.go:64] FLAG: --v="2" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769626 2570 flags.go:64] FLAG: --version="false" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769630 2570 flags.go:64] FLAG: --vmodule="" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769635 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.769638 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769730 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769734 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769737 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769740 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769743 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:51:52.774290 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769748 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769752 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769755 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769758 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769761 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769765 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769768 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769771 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769775 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769777 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769780 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769783 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769786 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769788 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769791 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769794 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769797 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769800 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769802 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769807 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:51:52.774870 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769809 2570 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769812 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769815 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769818 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769820 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769823 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769825 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769828 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769830 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769833 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769836 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769838 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769841 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769843 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769846 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769849 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769851 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769854 2570 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769864 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769868 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:51:52.775376 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769870 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769874 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769877 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769879 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769883 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769886 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769889 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769891 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769893 2570 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769896 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769899 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769902 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769905 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769907 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769910 2570 feature_gate.go:328] unrecognized feature gate: Example May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769913 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769915 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769918 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769920 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769923 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:51:52.775880 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769925 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769928 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769930 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769933 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769936 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769938 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769941 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769943 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769945 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769948 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769950 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769953 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769956 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769959 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769963 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769966 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769969 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769972 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769975 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:51:52.776371 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769977 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:51:52.777206 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.769979 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:51:52.777206 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.770893 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:51:52.778101 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.778079 2570 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 28 20:51:52.778101 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.778100 2570 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778167 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778176 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778181 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778186 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778191 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778195 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778200 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778204 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778208 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778212 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778217 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778221 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778225 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778230 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778234 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778238 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778242 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:51:52.778237 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778246 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778251 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778255 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778259 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778263 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778269 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778273 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778277 2570 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778282 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778287 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778291 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778295 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778298 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778303 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778307 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778311 2570 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778316 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778320 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778324 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778328 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:51:52.779064 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778332 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778337 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778341 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778344 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778348 2570 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778352 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778356 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778362 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778369 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778373 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778378 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778382 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778386 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778390 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778395 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778399 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778406 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778412 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778417 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:51:52.779671 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778422 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778427 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778433 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778438 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778442 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778447 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778451 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778456 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778460 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778464 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778490 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778495 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778511 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778516 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778520 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778524 2570 feature_gate.go:328] unrecognized feature gate: Example May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778528 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778533 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778537 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778541 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:51:52.780264 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778545 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778550 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778553 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778557 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778562 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778566 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778570 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778575 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778581 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778585 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.778593 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778749 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778758 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778764 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778769 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 28 20:51:52.780908 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778775 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778779 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778783 2570 feature_gate.go:328] unrecognized feature gate: DualReplica May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778788 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778792 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778796 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778800 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778804 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778809 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778812 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778817 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778821 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778825 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778829 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778833 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778837 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778841 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778845 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778849 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778853 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 28 20:51:52.781570 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778857 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778861 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778865 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778869 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778873 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778878 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778882 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778886 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778890 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778895 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778899 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778903 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778907 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778912 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778916 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778920 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778924 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778928 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778932 2570 feature_gate.go:328] unrecognized feature gate: Example May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778936 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 28 20:51:52.782354 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778940 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778944 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778948 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778953 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778957 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778961 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778965 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778969 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778973 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778977 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778981 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778987 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778993 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.778997 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779001 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779006 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779009 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779014 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779019 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 28 20:51:52.782930 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779023 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779028 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779033 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779037 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779041 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779045 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779049 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779054 2570 feature_gate.go:328] unrecognized feature gate: NewOLM May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779058 2570 feature_gate.go:328] unrecognized feature gate: Example2 May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779062 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779066 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779070 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779074 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779078 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779082 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779086 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779090 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779095 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779099 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779103 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 28 20:51:52.783511 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779107 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 28 20:51:52.783999 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779111 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 28 20:51:52.783999 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:52.779116 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 28 20:51:52.783999 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.779123 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 28 20:51:52.783999 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.779968 2570 server.go:962] "Client rotation is on, will bootstrap in background" May 28 20:51:52.783999 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.782970 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 28 20:51:52.784185 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.784172 2570 server.go:1019] "Starting client certificate rotation" May 28 20:51:52.784296 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.784280 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 20:51:52.784333 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.784317 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 28 20:51:52.812995 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.812969 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 20:51:52.817905 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.817887 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 28 20:51:52.835952 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.835928 2570 log.go:25] "Validated CRI v1 runtime API" May 28 20:51:52.844188 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.844167 2570 log.go:25] "Validated CRI v1 image API" May 28 20:51:52.845504 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.845489 2570 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 28 20:51:52.846922 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.846900 2570 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 20:51:52.848062 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.848038 2570 fs.go:135] Filesystem UUIDs: map[0bec85af-31f4-42f1-b0c3-73d0a1cbad88:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c953be38-b96b-4712-a1a6-11736cbb4555:/dev/nvme0n1p4] May 28 20:51:52.848119 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.848063 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 28 20:51:52.852388 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.852283 2570 manager.go:217] Machine: {Timestamp:2026-05-28 20:51:52.851805955 +0000 UTC m=+0.467640666 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104269 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d60f6f31f26c198bec9a9357f6522 SystemUUID:ec2d60f6-f31f-26c1-98be-c9a9357f6522 BootID:b010009d-64b5-4c8b-a32f-e621a0e6e24f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:36:b4:5b:e6:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:36:b4:5b:e6:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:19:c9:5f:5d:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 28 20:51:52.852388 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.852386 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 28 20:51:52.852520 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.852461 2570 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 28 20:51:52.855546 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.855517 2570 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 28 20:51:52.855709 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.855549 2570 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-77.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 28 20:51:52.855790 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.855725 2570 topology_manager.go:138] "Creating topology manager with none policy" May 28 20:51:52.855790 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.855738 2570 container_manager_linux.go:306] "Creating device plugin manager" May 28 20:51:52.855790 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.855747 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 20:51:52.856707 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.856694 2570 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 28 20:51:52.858449 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.858436 2570 state_mem.go:36] "Initialized new in-memory state store" May 28 20:51:52.858593 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.858582 2570 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 28 20:51:52.861257 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.861245 2570 kubelet.go:491] "Attempting to sync node with API server" May 28 20:51:52.861321 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.861263 2570 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 28 20:51:52.861321 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.861279 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 28 20:51:52.861321 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.861293 2570 kubelet.go:397] "Adding apiserver pod source" May 28 20:51:52.861321 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.861306 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 28 20:51:52.862897 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.862883 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 20:51:52.862964 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.862908 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 28 20:51:52.866485 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.866460 2570 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 28 20:51:52.868560 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.868547 2570 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 28 20:51:52.869492 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869467 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 28 20:51:52.869527 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869499 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 28 20:51:52.869527 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869509 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 28 20:51:52.869527 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869519 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 28 20:51:52.869527 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869527 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869533 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869539 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869545 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869551 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869557 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 28 20:51:52.869629 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869564 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 28 20:51:52.869971 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869962 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 28 20:51:52.870004 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.869992 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 28 20:51:52.870004 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.870001 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 28 20:51:52.873101 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.873087 2570 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 28 20:51:52.873173 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.873121 2570 server.go:1295] "Started kubelet" May 28 20:51:52.873222 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.873177 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 28 20:51:52.873281 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.873226 2570 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 28 20:51:52.873326 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.873309 2570 server_v1.go:47] "podresources" method="list" useActivePods=true May 28 20:51:52.873984 ip-10-0-136-77 systemd[1]: Started Kubernetes Kubelet. May 28 20:51:52.874743 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.874659 2570 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 28 20:51:52.876373 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.876355 2570 server.go:317] "Adding debug handlers to kubelet server" May 28 20:51:52.881921 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.881899 2570 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-77.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 28 20:51:52.882005 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.881918 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 28 20:51:52.882075 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.882037 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 28 20:51:52.882938 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.882920 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 28 20:51:52.883061 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.883046 2570 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 28 20:51:52.883206 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.882070 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-77.ec2.internal.18b3d5f1f2758099 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-77.ec2.internal,UID:ip-10-0-136-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-77.ec2.internal,},FirstTimestamp:2026-05-28 20:51:52.873099417 +0000 UTC m=+0.488934129,LastTimestamp:2026-05-28 20:51:52.873099417 +0000 UTC m=+0.488934129,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-77.ec2.internal,}" May 28 20:51:52.883526 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883510 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 28 20:51:52.883654 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883639 2570 volume_manager.go:295] "The desired_state_of_world populator starts" May 28 20:51:52.883706 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883656 2570 volume_manager.go:297] "Starting Kubelet Volume Manager" May 28 20:51:52.883761 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883744 2570 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 28 20:51:52.883808 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883781 2570 reconstruct.go:97] "Volume reconstruction finished" May 28 20:51:52.883808 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.883786 2570 reconciler.go:26] "Reconciler: start to sync state" May 28 20:51:52.884369 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884353 2570 factory.go:55] Registering systemd factory May 28 20:51:52.884369 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884369 2570 factory.go:223] Registration of the systemd container factory successfully May 28 20:51:52.884593 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884579 2570 factory.go:153] Registering CRI-O factory May 28 20:51:52.884634 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884597 2570 factory.go:223] Registration of the crio container factory successfully May 28 20:51:52.884675 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884646 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 28 20:51:52.884675 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884669 2570 factory.go:103] Registering Raw factory May 28 20:51:52.884774 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.884683 2570 manager.go:1196] Started watching for new ooms in manager May 28 20:51:52.884774 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.884719 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:52.885209 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.885197 2570 manager.go:319] Starting recovery of all containers May 28 20:51:52.888577 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.886739 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" May 28 20:51:52.888577 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.886835 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-77.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" May 28 20:51:52.895684 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.895537 2570 manager.go:324] Recovery completed May 28 20:51:52.897305 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.897282 2570 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory May 28 20:51:52.900435 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.900424 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:52.902332 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902317 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:52.902396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902345 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:52.902396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902356 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:52.902876 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902858 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" May 28 20:51:52.902876 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902875 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 28 20:51:52.902987 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.902892 2570 state_mem.go:36] "Initialized new in-memory state store" May 28 20:51:52.905985 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.905921 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-77.ec2.internal.18b3d5f1f4338d17 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-77.ec2.internal,UID:ip-10-0-136-77.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-77.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-77.ec2.internal,},FirstTimestamp:2026-05-28 20:51:52.902331671 +0000 UTC m=+0.518166382,LastTimestamp:2026-05-28 20:51:52.902331671 +0000 UTC m=+0.518166382,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-77.ec2.internal,}" May 28 20:51:52.906455 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.906440 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8zr7" May 28 20:51:52.906660 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.906650 2570 policy_none.go:49] "None policy: Start" May 28 20:51:52.906699 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.906665 2570 memory_manager.go:186] "Starting memorymanager" policy="None" May 28 20:51:52.906699 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.906676 2570 state_mem.go:35] "Initializing new in-memory state store" May 28 20:51:52.913960 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.913945 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v8zr7" May 28 20:51:52.946080 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946068 2570 manager.go:341] "Starting Device Plugin manager" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.946105 2570 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946117 2570 server.go:85] "Starting device plugin registration server" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946580 2570 eviction_manager.go:189] "Eviction manager: starting control loop" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946597 2570 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946901 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946976 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.946985 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.947511 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 28 20:51:52.969587 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.947564 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:52.985323 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.985286 2570 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 28 20:51:52.986552 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.986529 2570 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 28 20:51:52.986632 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.986559 2570 status_manager.go:230] "Starting to sync pod status with apiserver" May 28 20:51:52.986632 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.986579 2570 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 28 20:51:52.986632 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.986587 2570 kubelet.go:2451] "Starting kubelet main sync loop" May 28 20:51:52.986632 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:52.986628 2570 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 28 20:51:52.989482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:52.989448 2570 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:51:53.047889 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.047822 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:53.048820 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.048803 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:53.048890 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.048836 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:53.048890 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.048851 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:53.048890 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.048875 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-77.ec2.internal" May 28 20:51:53.058869 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.058852 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-77.ec2.internal" May 28 20:51:53.058924 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.058874 2570 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-77.ec2.internal\": node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.085972 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.085950 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.086912 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.086899 2570 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal"] May 28 20:51:53.086960 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.086955 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:53.088534 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.088515 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:53.088606 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.088545 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:53.088606 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.088555 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:53.089770 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.089759 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:53.089924 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.089910 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.089981 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.089937 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:53.090917 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090903 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:53.090917 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090911 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:53.091066 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090928 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:53.091066 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090930 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:53.091066 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090968 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:53.091066 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.090942 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:53.092155 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.092139 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:53.092223 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.092165 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 28 20:51:53.092859 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.092840 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientMemory" May 28 20:51:53.092939 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.092867 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasNoDiskPressure" May 28 20:51:53.092939 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.092879 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeHasSufficientPID" May 28 20:51:53.121093 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.121075 2570 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-77.ec2.internal\" not found" node="ip-10-0-136-77.ec2.internal" May 28 20:51:53.125586 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.125570 2570 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-77.ec2.internal\" not found" node="ip-10-0-136-77.ec2.internal" May 28 20:51:53.186672 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.186651 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.284606 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.284579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.284721 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.284612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.284721 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.284633 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18bc8a3cba233c5956614825d898bb27-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"18bc8a3cba233c5956614825d898bb27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:53.287647 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.287633 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.385336 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385266 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.385336 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385314 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.385336 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385333 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18bc8a3cba233c5956614825d898bb27-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"18bc8a3cba233c5956614825d898bb27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:53.385458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385372 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.385458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385384 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/18bc8a3cba233c5956614825d898bb27-config\") pod \"kube-apiserver-proxy-ip-10-0-136-77.ec2.internal\" (UID: \"18bc8a3cba233c5956614825d898bb27\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:53.385458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.385400 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/811679a399699dc628e5f5db58e602a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal\" (UID: \"811679a399699dc628e5f5db58e602a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.388356 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.388340 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.425552 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.425519 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:53.428278 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.428249 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:53.489058 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.489030 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.589535 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.589503 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.690030 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.689963 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.784296 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.784265 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 28 20:51:53.784843 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.784415 2570 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 28 20:51:53.790577 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.790556 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.883822 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.883793 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 28 20:51:53.890645 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.890616 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.907962 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.907940 2570 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 28 20:51:53.916086 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.916017 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 20:46:52 +0000 UTC" deadline="2028-03-08 02:33:27.520167544 +0000 UTC" May 28 20:51:53.916086 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.916060 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15581h41m33.604111805s" May 28 20:51:53.939111 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.939087 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-thxvq" May 28 20:51:53.948061 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:53.948005 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-thxvq" May 28 20:51:53.991082 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:53.991064 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-136-77.ec2.internal\" not found" May 28 20:51:53.999057 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:53.999029 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bc8a3cba233c5956614825d898bb27.slice/crio-a9004e06442819acde8879fa2e2750ce69ca379afd573a17aa9ab99b6d9d0069 WatchSource:0}: Error finding container a9004e06442819acde8879fa2e2750ce69ca379afd573a17aa9ab99b6d9d0069: Status 404 returned error can't find the container with id a9004e06442819acde8879fa2e2750ce69ca379afd573a17aa9ab99b6d9d0069 May 28 20:51:53.999322 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:53.999302 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811679a399699dc628e5f5db58e602a6.slice/crio-68bec2aec40b4fb892b46ac93953cd47cb2ded79820da4135d94ed7c68f7689b WatchSource:0}: Error finding container 68bec2aec40b4fb892b46ac93953cd47cb2ded79820da4135d94ed7c68f7689b: Status 404 returned error can't find the container with id 68bec2aec40b4fb892b46ac93953cd47cb2ded79820da4135d94ed7c68f7689b May 28 20:51:54.002872 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.002859 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 20:51:54.049520 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.049486 2570 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:51:54.084143 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.084114 2570 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" May 28 20:51:54.099152 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.099127 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 20:51:54.101284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.101263 2570 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" May 28 20:51:54.109219 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.109194 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 28 20:51:54.317176 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.317098 2570 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:51:54.458521 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.458487 2570 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:51:54.735531 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.735436 2570 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 28 20:51:54.862631 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.862599 2570 apiserver.go:52] "Watching apiserver" May 28 20:51:54.868255 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.868225 2570 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 28 20:51:54.870557 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.870534 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-jstj6","openshift-image-registry/node-ca-hzr92","openshift-multus/network-metrics-daemon-mspmh","openshift-cluster-node-tuning-operator/tuned-xmjvg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal","openshift-multus/multus-additional-cni-plugins-p79nd","openshift-multus/multus-bxvvb","openshift-network-diagnostics/network-check-target-nh864","openshift-network-operator/iptables-alerter-m9fph","openshift-ovn-kubernetes/ovnkube-node-j8294","kube-system/konnectivity-agent-gtpfl","kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r"] May 28 20:51:54.874865 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.874846 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bxvvb" May 28 20:51:54.876856 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.876831 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 28 20:51:54.876856 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.876844 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.877024 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.876854 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xnnl8\"" May 28 20:51:54.877394 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.877371 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 28 20:51:54.877571 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.877511 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 28 20:51:54.877658 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.877633 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 28 20:51:54.879044 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.878973 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:54.879913 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:54.879042 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:51:54.881752 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.881588 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mwhzj\"" May 28 20:51:54.882631 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.882147 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 28 20:51:54.882631 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.882205 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.882631 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.882211 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 28 20:51:54.882631 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.882325 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 28 20:51:54.884834 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.884561 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.884834 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.884634 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 28 20:51:54.884834 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.884805 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 28 20:51:54.885045 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.884812 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ffz75\"" May 28 20:51:54.886056 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.886035 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wf6w9\"" May 28 20:51:54.886735 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.886716 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.886842 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.886722 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 28 20:51:54.887225 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.887208 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 28 20:51:54.888361 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.888320 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 28 20:51:54.888466 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.888450 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 28 20:51:54.888557 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.888452 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zcl57\"" May 28 20:51:54.889557 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.889540 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:54.889655 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:54.889606 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:51:54.889655 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.889648 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:54.891035 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.891015 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 28 20:51:54.891138 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.891124 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 28 20:51:54.891194 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.891129 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 28 20:51:54.891411 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.891395 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-blvfp\"" May 28 20:51:54.891952 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.891933 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.892635 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-os-release\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.892711 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9d377526-7f51-4275-ad9e-c99a0a1766ed-serviceca\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.892770 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892712 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-conf\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.892770 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-run\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.892770 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892765 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-tuned\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.892914 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892791 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-os-release\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.892914 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-conf-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.892914 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892839 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:54.892914 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892880 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysconfig\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893104 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892914 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.893104 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.893104 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntd25\" (UniqueName: \"kubernetes.io/projected/2d2d247b-766d-4b6f-845f-0d4a22141dc7-kube-api-access-ntd25\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.893104 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.892985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-cnibin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893104 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-etc-kubernetes\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-lib-modules\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893128 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cnibin\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-bin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893208 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-multus\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893224 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d377526-7f51-4275-ad9e-c99a0a1766ed-host\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893250 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-cni-binary-copy\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893285 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4qgsc\"" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893293 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brh56\" (UniqueName: \"kubernetes.io/projected/9d377526-7f51-4275-ad9e-c99a0a1766ed-kube-api-access-brh56\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.893339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893313 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-modprobe-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893352 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893386 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-systemd\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893409 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-var-lib-kubelet\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bed4256-2823-4ff3-a5e1-dabf5429faa7-tmp-dir\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893495 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893531 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-netns\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-kubelet\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893587 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-daemon-config\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893611 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-multus-certs\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893627 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893632 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-host\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-socket-dir-parent\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893741 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr79n\" (UniqueName: \"kubernetes.io/projected/1bed4256-2823-4ff3-a5e1-dabf5429faa7-kube-api-access-gr79n\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893767 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-system-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.893806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893792 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-hostroot\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kks\" (UniqueName: \"kubernetes.io/projected/52d66fbd-189a-41e1-a106-06795d3fe62c-kube-api-access-p7kks\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893841 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-kubernetes\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893886 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-tmp\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-system-cni-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893950 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bed4256-2823-4ff3-a5e1-dabf5429faa7-hosts-file\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.893985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-k8s-cni-cncf-io\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894011 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnhn\" (UniqueName: \"kubernetes.io/projected/df288737-cf68-4b61-9f6b-4a87f5df1a98-kube-api-access-kvnhn\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894043 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-sys\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894067 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsj2\" (UniqueName: \"kubernetes.io/projected/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-kube-api-access-vfsj2\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894094 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.894603 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894443 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 28 20:51:54.895140 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894618 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 28 20:51:54.895140 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894686 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 28 20:51:54.895140 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.894621 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 28 20:51:54.895270 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.895244 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:54.896913 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.896895 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c94qv\"" May 28 20:51:54.897003 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.896971 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 28 20:51:54.897068 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.897009 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 28 20:51:54.897611 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.897594 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.899045 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.899025 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 28 20:51:54.900051 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.899373 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 28 20:51:54.900051 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.899508 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 28 20:51:54.900051 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.899754 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-289j5\"" May 28 20:51:54.948649 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.948619 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 20:46:53 +0000 UTC" deadline="2028-01-08 11:58:26.576187093 +0000 UTC" May 28 20:51:54.948649 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.948643 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14151h6m31.627546276s" May 28 20:51:54.985300 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.985274 2570 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 28 20:51:54.990906 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.990820 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" event={"ID":"18bc8a3cba233c5956614825d898bb27","Type":"ContainerStarted","Data":"a9004e06442819acde8879fa2e2750ce69ca379afd573a17aa9ab99b6d9d0069"} May 28 20:51:54.992219 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.992198 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"811679a399699dc628e5f5db58e602a6","Type":"ContainerStarted","Data":"68bec2aec40b4fb892b46ac93953cd47cb2ded79820da4135d94ed7c68f7689b"} May 28 20:51:54.994464 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994443 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-slash\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.994578 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994493 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-cni-binary-copy\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994578 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994518 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-brh56\" (UniqueName: \"kubernetes.io/projected/9d377526-7f51-4275-ad9e-c99a0a1766ed-kube-api-access-brh56\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.994578 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994534 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-modprobe-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.994578 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994550 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.994755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994676 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-modprobe-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.994755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994682 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bed4256-2823-4ff3-a5e1-dabf5429faa7-tmp-dir\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.994755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994722 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994748 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-netns\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994773 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-multus-certs\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994778 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-d\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994796 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-sys-fs\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994839 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994851 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-socket-dir-parent\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994872 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-multus-certs\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994878 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gr79n\" (UniqueName: \"kubernetes.io/projected/1bed4256-2823-4ff3-a5e1-dabf5429faa7-kube-api-access-gr79n\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.994908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994902 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-hostroot\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994923 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-netns\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994929 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kks\" (UniqueName: \"kubernetes.io/projected/52d66fbd-189a-41e1-a106-06795d3fe62c-kube-api-access-p7kks\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.994956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995001 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/430ffa20-7a97-44d5-b166-bf23ed6c48f3-iptables-alerter-script\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995027 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-k8s-cni-cncf-io\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995050 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsj2\" (UniqueName: \"kubernetes.io/projected/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-kube-api-access-vfsj2\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f654f73d-9fd6-4361-b611-d853c545123f-konnectivity-ca\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995079 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995096 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bed4256-2823-4ff3-a5e1-dabf5429faa7-tmp-dir\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovn-node-metrics-cert\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-socket-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995153 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-cni-binary-copy\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995167 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9d377526-7f51-4275-ad9e-c99a0a1766ed-serviceca\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.995203 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995199 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-conf\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995228 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-tuned\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995252 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-os-release\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995253 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-run-k8s-cni-cncf-io\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995280 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995336 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-socket-dir-parent\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995403 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-conf-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995451 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-conf-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995484 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995503 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-hostroot\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995520 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysconfig\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995558 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995570 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysconfig\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995589 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ntd25\" (UniqueName: \"kubernetes.io/projected/2d2d247b-766d-4b6f-845f-0d4a22141dc7-kube-api-access-ntd25\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9d377526-7f51-4275-ad9e-c99a0a1766ed-serviceca\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:54.995621 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:54.995716 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:55.495677463 +0000 UTC m=+3.111512184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:54.995767 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995734 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995747 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-os-release\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47q8f\" (UniqueName: \"kubernetes.io/projected/8c8228ec-5064-45e2-bee8-aaa03fe0691c-kube-api-access-47q8f\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-sysctl-conf\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995869 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-cnibin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995911 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-node-log\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995917 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-cnibin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-log-socket\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-config\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.995993 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/430ffa20-7a97-44d5-b166-bf23ed6c48f3-host-slash\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996027 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-bin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996045 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996050 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-multus\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996087 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-multus\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996100 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d377526-7f51-4275-ad9e-c99a0a1766ed-host\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.996482 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996144 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d377526-7f51-4275-ad9e-c99a0a1766ed-host\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-kubelet\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996100 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-cni-bin\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996190 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-systemd\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996231 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-systemd\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996262 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-var-lib-kubelet\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996293 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-host\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996307 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-var-lib-kubelet\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996333 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6l2s\" (UniqueName: \"kubernetes.io/projected/430ffa20-7a97-44d5-b166-bf23ed6c48f3-kube-api-access-n6l2s\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996348 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-host\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996358 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-kubelet\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996389 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-daemon-config\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996411 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-ovn\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996420 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-host-var-lib-kubelet\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-device-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996521 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-system-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996541 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-kubernetes\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996563 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-tmp\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996607 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-system-cni-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-var-lib-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996665 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bed4256-2823-4ff3-a5e1-dabf5429faa7-hosts-file\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-kubernetes\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1bed4256-2823-4ff3-a5e1-dabf5429faa7-hosts-file\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996737 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-system-cni-dir\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-system-cni-dir\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996768 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnhn\" (UniqueName: \"kubernetes.io/projected/df288737-cf68-4b61-9f6b-4a87f5df1a98-kube-api-access-kvnhn\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996795 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-sys\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996821 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996849 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f654f73d-9fd6-4361-b611-d853c545123f-agent-certs\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-systemd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996907 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/52d66fbd-189a-41e1-a106-06795d3fe62c-multus-daemon-config\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-bin\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996963 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-sys\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.996965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-registration-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.997950 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997014 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-os-release\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997039 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-run\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997082 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997124 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-os-release\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997139 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-run\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997189 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-etc-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997220 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-etc-kubernetes\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997248 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-lib-modules\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997272 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cnibin\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997286 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52d66fbd-189a-41e1-a106-06795d3fe62c-etc-kubernetes\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997299 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-systemd-units\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997324 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-netd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997352 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997378 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-env-overrides\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997380 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cni-binary-copy\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997398 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-lib-modules\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997402 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qkm\" (UniqueName: \"kubernetes.io/projected/bf6f9d86-22a2-4662-860d-fc00e48372a6-kube-api-access-v6qkm\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:54.998505 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997428 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2d2d247b-766d-4b6f-845f-0d4a22141dc7-cnibin\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.999155 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-netns\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.999155 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-script-lib\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:54.999155 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.997582 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2d2d247b-766d-4b6f-845f-0d4a22141dc7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:54.999361 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.999340 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-tmp\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:54.999422 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:54.999393 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-etc-tuned\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:55.006130 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.006102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsj2\" (UniqueName: \"kubernetes.io/projected/938ef7dc-3b01-4087-ae13-1c9da2b19ee2-kube-api-access-vfsj2\") pod \"tuned-xmjvg\" (UID: \"938ef7dc-3b01-4087-ae13-1c9da2b19ee2\") " pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:55.007445 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.007305 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr79n\" (UniqueName: \"kubernetes.io/projected/1bed4256-2823-4ff3-a5e1-dabf5429faa7-kube-api-access-gr79n\") pod \"node-resolver-jstj6\" (UID: \"1bed4256-2823-4ff3-a5e1-dabf5429faa7\") " pod="openshift-dns/node-resolver-jstj6" May 28 20:51:55.008141 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.007699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntd25\" (UniqueName: \"kubernetes.io/projected/2d2d247b-766d-4b6f-845f-0d4a22141dc7-kube-api-access-ntd25\") pod \"multus-additional-cni-plugins-p79nd\" (UID: \"2d2d247b-766d-4b6f-845f-0d4a22141dc7\") " pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:55.008141 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.008080 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brh56\" (UniqueName: \"kubernetes.io/projected/9d377526-7f51-4275-ad9e-c99a0a1766ed-kube-api-access-brh56\") pod \"node-ca-hzr92\" (UID: \"9d377526-7f51-4275-ad9e-c99a0a1766ed\") " pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:55.008382 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.008361 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kks\" (UniqueName: \"kubernetes.io/projected/52d66fbd-189a-41e1-a106-06795d3fe62c-kube-api-access-p7kks\") pod \"multus-bxvvb\" (UID: \"52d66fbd-189a-41e1-a106-06795d3fe62c\") " pod="openshift-multus/multus-bxvvb" May 28 20:51:55.011599 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.011574 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnhn\" (UniqueName: \"kubernetes.io/projected/df288737-cf68-4b61-9f6b-4a87f5df1a98-kube-api-access-kvnhn\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:55.098273 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098241 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098281 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/430ffa20-7a97-44d5-b166-bf23ed6c48f3-iptables-alerter-script\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098305 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f654f73d-9fd6-4361-b611-d853c545123f-konnectivity-ca\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098325 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovn-node-metrics-cert\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098348 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-socket-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098370 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098376 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.098433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098414 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098490 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098504 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098527 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-47q8f\" (UniqueName: \"kubernetes.io/projected/8c8228ec-5064-45e2-bee8-aaa03fe0691c-kube-api-access-47q8f\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098569 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-node-log\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098549 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-socket-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098610 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-node-log\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098596 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-log-socket\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098655 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-config\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098677 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-log-socket\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098684 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/430ffa20-7a97-44d5-b166-bf23ed6c48f3-host-slash\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098712 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098737 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/430ffa20-7a97-44d5-b166-bf23ed6c48f3-host-slash\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098744 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:55.098784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098775 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-kubelet\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098804 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n6l2s\" (UniqueName: \"kubernetes.io/projected/430ffa20-7a97-44d5-b166-bf23ed6c48f3-kube-api-access-n6l2s\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098830 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-ovn\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098860 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-etc-selinux\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098880 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-ovn\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098884 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-kubelet\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098923 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-device-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098960 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-var-lib-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098974 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f654f73d-9fd6-4361-b611-d853c545123f-konnectivity-ca\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.098991 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f654f73d-9fd6-4361-b611-d853c545123f-agent-certs\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099017 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-systemd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099033 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-device-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099042 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-bin\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099067 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-registration-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099076 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-var-lib-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099108 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-etc-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099115 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-run-systemd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.099487 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099142 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-systemd-units\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099175 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-registration-dir\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099179 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-netd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099178 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-bin\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099214 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099222 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-systemd-units\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099230 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-config\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099242 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-env-overrides\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099225 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-cni-netd\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099272 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qkm\" (UniqueName: \"kubernetes.io/projected/bf6f9d86-22a2-4662-860d-fc00e48372a6-kube-api-access-v6qkm\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099290 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-etc-openvswitch\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099297 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-netns\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099304 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099322 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-script-lib\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099347 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-run-netns\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099348 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-slash\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099381 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c8228ec-5064-45e2-bee8-aaa03fe0691c-host-slash\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.100246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099393 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-sys-fs\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.101022 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099918 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/430ffa20-7a97-44d5-b166-bf23ed6c48f3-iptables-alerter-script\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.101022 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.099876 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bf6f9d86-22a2-4662-860d-fc00e48372a6-sys-fs\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.101022 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.100183 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-env-overrides\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.101022 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.100310 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovnkube-script-lib\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.103058 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.101609 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f654f73d-9fd6-4361-b611-d853c545123f-agent-certs\") pod \"konnectivity-agent-gtpfl\" (UID: \"f654f73d-9fd6-4361-b611-d853c545123f\") " pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:55.103369 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.103347 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8c8228ec-5064-45e2-bee8-aaa03fe0691c-ovn-node-metrics-cert\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.110199 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.110177 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:51:55.110199 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.110201 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:51:55.110330 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.110212 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:55.110330 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.110271 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:55.610253282 +0000 UTC m=+3.226087997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:55.112606 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.112587 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qkm\" (UniqueName: \"kubernetes.io/projected/bf6f9d86-22a2-4662-860d-fc00e48372a6-kube-api-access-v6qkm\") pod \"aws-ebs-csi-driver-node-xrb7r\" (UID: \"bf6f9d86-22a2-4662-860d-fc00e48372a6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.112823 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.112805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6l2s\" (UniqueName: \"kubernetes.io/projected/430ffa20-7a97-44d5-b166-bf23ed6c48f3-kube-api-access-n6l2s\") pod \"iptables-alerter-m9fph\" (UID: \"430ffa20-7a97-44d5-b166-bf23ed6c48f3\") " pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.112888 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.112861 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47q8f\" (UniqueName: \"kubernetes.io/projected/8c8228ec-5064-45e2-bee8-aaa03fe0691c-kube-api-access-47q8f\") pod \"ovnkube-node-j8294\" (UID: \"8c8228ec-5064-45e2-bee8-aaa03fe0691c\") " pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.189173 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.189139 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bxvvb" May 28 20:51:55.197022 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.196996 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hzr92" May 28 20:51:55.205691 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.205669 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jstj6" May 28 20:51:55.209250 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.209231 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" May 28 20:51:55.215169 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.215150 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p79nd" May 28 20:51:55.224615 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.224593 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9fph" May 28 20:51:55.231159 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.231143 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:51:55.236784 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.236710 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:51:55.241250 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.241201 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" May 28 20:51:55.501119 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.501033 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:55.501277 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.501201 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:55.501346 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.501280 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:56.501258455 +0000 UTC m=+4.117093152 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:55.600180 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.600152 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430ffa20_7a97_44d5_b166_bf23ed6c48f3.slice/crio-870d33b1eb0180eacd902047f927b1ed2fbfb51e6d4e3a65e1c914e9d6a9bceb WatchSource:0}: Error finding container 870d33b1eb0180eacd902047f927b1ed2fbfb51e6d4e3a65e1c914e9d6a9bceb: Status 404 returned error can't find the container with id 870d33b1eb0180eacd902047f927b1ed2fbfb51e6d4e3a65e1c914e9d6a9bceb May 28 20:51:55.600933 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.600803 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf6f9d86_22a2_4662_860d_fc00e48372a6.slice/crio-0665c8b5898da43b053a835c9f8f00c45c67d8951e8b1ed1c6bb2d33dc1392fa WatchSource:0}: Error finding container 0665c8b5898da43b053a835c9f8f00c45c67d8951e8b1ed1c6bb2d33dc1392fa: Status 404 returned error can't find the container with id 0665c8b5898da43b053a835c9f8f00c45c67d8951e8b1ed1c6bb2d33dc1392fa May 28 20:51:55.602333 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.602313 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf654f73d_9fd6_4361_b611_d853c545123f.slice/crio-c012cc3f966ca28c59fa75d60d990c412116ff6fdd704e6160dd256954545c5b WatchSource:0}: Error finding container c012cc3f966ca28c59fa75d60d990c412116ff6fdd704e6160dd256954545c5b: Status 404 returned error can't find the container with id c012cc3f966ca28c59fa75d60d990c412116ff6fdd704e6160dd256954545c5b May 28 20:51:55.606817 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.605010 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bed4256_2823_4ff3_a5e1_dabf5429faa7.slice/crio-d007f52b2d17fd40cbf891234018689a574d48b5f34392ad31e79e42de5c522b WatchSource:0}: Error finding container d007f52b2d17fd40cbf891234018689a574d48b5f34392ad31e79e42de5c522b: Status 404 returned error can't find the container with id d007f52b2d17fd40cbf891234018689a574d48b5f34392ad31e79e42de5c522b May 28 20:51:55.606817 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.605808 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52d66fbd_189a_41e1_a106_06795d3fe62c.slice/crio-7ace389b00c48a67fae7bcc13b0ae9873211e3fcc9fe709a627e93100d281376 WatchSource:0}: Error finding container 7ace389b00c48a67fae7bcc13b0ae9873211e3fcc9fe709a627e93100d281376: Status 404 returned error can't find the container with id 7ace389b00c48a67fae7bcc13b0ae9873211e3fcc9fe709a627e93100d281376 May 28 20:51:55.607351 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.607250 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2d247b_766d_4b6f_845f_0d4a22141dc7.slice/crio-fb6f8341d533201331d91aa9d885c2739c292280b565d4497a5b02ab2b619d82 WatchSource:0}: Error finding container fb6f8341d533201331d91aa9d885c2739c292280b565d4497a5b02ab2b619d82: Status 404 returned error can't find the container with id fb6f8341d533201331d91aa9d885c2739c292280b565d4497a5b02ab2b619d82 May 28 20:51:55.607582 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.607550 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d377526_7f51_4275_ad9e_c99a0a1766ed.slice/crio-7311a711706f8d8cdc988933e94e6e7e152d06266b20f407fc14ab07346bd6c6 WatchSource:0}: Error finding container 7311a711706f8d8cdc988933e94e6e7e152d06266b20f407fc14ab07346bd6c6: Status 404 returned error can't find the container with id 7311a711706f8d8cdc988933e94e6e7e152d06266b20f407fc14ab07346bd6c6 May 28 20:51:55.608849 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.608779 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8228ec_5064_45e2_bee8_aaa03fe0691c.slice/crio-b1bfacb3a83043bfe399bb766b38b62898425f1a9bd6b684b8fbddca744eb2aa WatchSource:0}: Error finding container b1bfacb3a83043bfe399bb766b38b62898425f1a9bd6b684b8fbddca744eb2aa: Status 404 returned error can't find the container with id b1bfacb3a83043bfe399bb766b38b62898425f1a9bd6b684b8fbddca744eb2aa May 28 20:51:55.612035 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:51:55.611902 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod938ef7dc_3b01_4087_ae13_1c9da2b19ee2.slice/crio-f9f7be7213c554f9d4df90172116fdfc09bb06e01acdcaad3c0871ad5e496347 WatchSource:0}: Error finding container f9f7be7213c554f9d4df90172116fdfc09bb06e01acdcaad3c0871ad5e496347: Status 404 returned error can't find the container with id f9f7be7213c554f9d4df90172116fdfc09bb06e01acdcaad3c0871ad5e496347 May 28 20:51:55.702520 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.702377 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:55.702614 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.702540 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:51:55.702614 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.702558 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:51:55.702614 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.702567 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:55.702614 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:55.702612 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:56.702599168 +0000 UTC m=+4.318433867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:55.949624 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.949585 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 20:46:53 +0000 UTC" deadline="2028-01-17 15:44:14.688394551 +0000 UTC" May 28 20:51:55.949624 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.949619 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14370h52m18.738778121s" May 28 20:51:55.995644 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.995607 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jstj6" event={"ID":"1bed4256-2823-4ff3-a5e1-dabf5429faa7","Type":"ContainerStarted","Data":"d007f52b2d17fd40cbf891234018689a574d48b5f34392ad31e79e42de5c522b"} May 28 20:51:55.997341 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.997313 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gtpfl" event={"ID":"f654f73d-9fd6-4361-b611-d853c545123f","Type":"ContainerStarted","Data":"c012cc3f966ca28c59fa75d60d990c412116ff6fdd704e6160dd256954545c5b"} May 28 20:51:55.998523 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:55.998496 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9fph" event={"ID":"430ffa20-7a97-44d5-b166-bf23ed6c48f3","Type":"ContainerStarted","Data":"870d33b1eb0180eacd902047f927b1ed2fbfb51e6d4e3a65e1c914e9d6a9bceb"} May 28 20:51:56.001307 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.001278 2570 generic.go:358] "Generic (PLEG): container finished" podID="811679a399699dc628e5f5db58e602a6" containerID="7dd3b51d500f7606176d30c9984cd285ad72722a656b503040fad9e1741167ab" exitCode=0 May 28 20:51:56.001398 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.001361 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"811679a399699dc628e5f5db58e602a6","Type":"ContainerDied","Data":"7dd3b51d500f7606176d30c9984cd285ad72722a656b503040fad9e1741167ab"} May 28 20:51:56.003266 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.003242 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"b1bfacb3a83043bfe399bb766b38b62898425f1a9bd6b684b8fbddca744eb2aa"} May 28 20:51:56.005985 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.005944 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hzr92" event={"ID":"9d377526-7f51-4275-ad9e-c99a0a1766ed","Type":"ContainerStarted","Data":"7311a711706f8d8cdc988933e94e6e7e152d06266b20f407fc14ab07346bd6c6"} May 28 20:51:56.007461 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.007432 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerStarted","Data":"fb6f8341d533201331d91aa9d885c2739c292280b565d4497a5b02ab2b619d82"} May 28 20:51:56.011260 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.011029 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" event={"ID":"bf6f9d86-22a2-4662-860d-fc00e48372a6","Type":"ContainerStarted","Data":"0665c8b5898da43b053a835c9f8f00c45c67d8951e8b1ed1c6bb2d33dc1392fa"} May 28 20:51:56.014730 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.014306 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" event={"ID":"18bc8a3cba233c5956614825d898bb27","Type":"ContainerStarted","Data":"b80586d59e1d2a10e4b2266e588a3efcd3c7a060f029806fdc366058691c45c4"} May 28 20:51:56.016003 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.015957 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" event={"ID":"938ef7dc-3b01-4087-ae13-1c9da2b19ee2","Type":"ContainerStarted","Data":"f9f7be7213c554f9d4df90172116fdfc09bb06e01acdcaad3c0871ad5e496347"} May 28 20:51:56.018789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.018750 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bxvvb" event={"ID":"52d66fbd-189a-41e1-a106-06795d3fe62c","Type":"ContainerStarted","Data":"7ace389b00c48a67fae7bcc13b0ae9873211e3fcc9fe709a627e93100d281376"} May 28 20:51:56.031246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.030520 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-77.ec2.internal" podStartSLOduration=2.030504602 podStartE2EDuration="2.030504602s" podCreationTimestamp="2026-05-28 20:51:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:51:56.030384041 +0000 UTC m=+3.646218762" watchObservedRunningTime="2026-05-28 20:51:56.030504602 +0000 UTC m=+3.646339324" May 28 20:51:56.508066 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.507950 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:56.508228 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.508110 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:56.508228 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.508180 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:58.508159632 +0000 UTC m=+6.123994347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:56.710196 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.710157 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:56.710408 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.710389 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:51:56.710498 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.710416 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:51:56.710498 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.710431 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:56.711172 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.711139 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:51:58.710484915 +0000 UTC m=+6.326319628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:56.997710 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.997627 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:56.998153 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.997810 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:51:56.998325 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:56.998306 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:56.998459 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:56.998431 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:51:57.049605 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:57.049548 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" event={"ID":"811679a399699dc628e5f5db58e602a6","Type":"ContainerStarted","Data":"14a166afc63225f85e80d33f7fc31deea33565c5a72e70fe2f5771622be2d8ee"} May 28 20:51:57.063531 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:57.063463 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-77.ec2.internal" podStartSLOduration=3.063446163 podStartE2EDuration="3.063446163s" podCreationTimestamp="2026-05-28 20:51:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:51:57.062844844 +0000 UTC m=+4.678679566" watchObservedRunningTime="2026-05-28 20:51:57.063446163 +0000 UTC m=+4.679280884" May 28 20:51:58.526329 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:58.525684 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:58.526329 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.525870 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:58.526329 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.525942 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:02.525923035 +0000 UTC m=+10.141757739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:51:58.727061 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:58.727010 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:58.727230 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.727174 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:51:58.727230 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.727192 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:51:58.727230 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.727206 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:58.727401 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.727258 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:02.727239779 +0000 UTC m=+10.343074476 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:51:58.988280 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:58.987573 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:51:58.988280 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.987693 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:51:58.988280 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:51:58.988067 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:51:58.988280 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:51:58.988167 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:00.987713 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:00.987609 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:00.988143 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:00.987743 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:00.988143 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:00.987791 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:00.988143 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:00.987888 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:02.558735 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:02.558706 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:02.559284 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.558845 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:02.559284 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.558916 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:10.558896363 +0000 UTC m=+18.174731074 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:02.760252 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:02.760218 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:02.760433 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.760393 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:02.760433 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.760410 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:02.760433 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.760421 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:02.760618 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.760502 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:10.760465848 +0000 UTC m=+18.376300553 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:02.988160 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:02.988073 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:02.988315 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.988185 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:02.988315 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:02.988267 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:02.988442 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:02.988377 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:04.987581 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:04.987553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:04.988020 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:04.987553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:04.988020 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:04.987672 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:04.988020 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:04.987717 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:06.987730 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:06.987694 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:06.987730 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:06.987694 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:06.988238 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:06.987890 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:06.988238 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:06.987921 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:08.987802 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:08.987767 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:08.988258 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:08.987767 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:08.988258 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:08.987884 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:08.988258 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:08.987978 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:10.612876 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:10.612836 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:10.613359 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.612962 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:10.613359 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.613027 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:26.613011795 +0000 UTC m=+34.228846493 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:10.813624 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:10.813589 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:10.813811 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.813788 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:10.813854 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.813817 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:10.813854 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.813831 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:10.813926 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.813898 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:26.8138726 +0000 UTC m=+34.429707304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:10.987626 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:10.987553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:10.987785 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:10.987553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:10.987785 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.987694 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:10.987785 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:10.987722 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:12.988254 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:12.988217 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:12.988693 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:12.988336 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:12.988693 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:12.988396 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:12.988693 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:12.988540 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:14.987820 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:14.987648 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:14.988377 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:14.987649 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:14.988377 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:14.987886 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:14.988377 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:14.987966 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:15.081319 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.081290 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="34da802795af47d74cd34527837448af2ae06d99152e13f9e2c0c345615a8b75" exitCode=0 May 28 20:52:15.081419 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.081383 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"34da802795af47d74cd34527837448af2ae06d99152e13f9e2c0c345615a8b75"} May 28 20:52:15.082977 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.082832 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" event={"ID":"bf6f9d86-22a2-4662-860d-fc00e48372a6","Type":"ContainerStarted","Data":"95a48c7ba3ca5a313510f0251788e32b5ceccd5da164e7d7429e7523fd538076"} May 28 20:52:15.084133 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.084108 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" event={"ID":"938ef7dc-3b01-4087-ae13-1c9da2b19ee2","Type":"ContainerStarted","Data":"3d1e22f972739abf17107904e8bdc9b5c0ba9f5f025b6ba7b8a067e7fe30633c"} May 28 20:52:15.085662 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.085639 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bxvvb" event={"ID":"52d66fbd-189a-41e1-a106-06795d3fe62c","Type":"ContainerStarted","Data":"df138fb3cbdaaf0c6f9d1f5ce0f1aa89654088e39145371c8f5e1de06c54c986"} May 28 20:52:15.086882 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.086864 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jstj6" event={"ID":"1bed4256-2823-4ff3-a5e1-dabf5429faa7","Type":"ContainerStarted","Data":"4ff65ced50819549ac548804164d9fff3fb7e8f489100cd14e313b8b9c7895c2"} May 28 20:52:15.088028 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.088002 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gtpfl" event={"ID":"f654f73d-9fd6-4361-b611-d853c545123f","Type":"ContainerStarted","Data":"f47c70656ed5617e070b5e0ee9d1a66eaea3c48a828a14873a31f09d5b8a4e5d"} May 28 20:52:15.090430 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.090404 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"9976189e6970cb3ad37cfedcc53470b1e673287d0de0ea63f362781ef8136d4f"} May 28 20:52:15.090430 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.090427 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"21d67a4968efe00fcd4ac9c78cb7c630f8ddfca86b3db6282b5b9eacc932364e"} May 28 20:52:15.090556 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.090436 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"53c34fb3486d4496fa5237e5da9f7ebbee641e0e30440a6e83b74ccf24ad0e58"} May 28 20:52:15.090556 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.090445 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"cf8b308c86c39dacb5fe6eca3d2c720ff1acda0205142637ffcc4d0215e723a0"} May 28 20:52:15.090556 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.090454 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"0a372609768e6104a125cf49cd96d253c1b3bcea125e5be19621b3d9d9d7b9b4"} May 28 20:52:15.091542 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.091525 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hzr92" event={"ID":"9d377526-7f51-4275-ad9e-c99a0a1766ed","Type":"ContainerStarted","Data":"5bbd0ea9812575a3590243582d2665724c4778e8c442a4ccbb3d145720faa53e"} May 28 20:52:15.143991 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.143899 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-bxvvb" podStartSLOduration=3.663623784 podStartE2EDuration="22.143878529s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.608121813 +0000 UTC m=+3.223956514" lastFinishedPulling="2026-05-28 20:52:14.088376546 +0000 UTC m=+21.704211259" observedRunningTime="2026-05-28 20:52:15.143273807 +0000 UTC m=+22.759108527" watchObservedRunningTime="2026-05-28 20:52:15.143878529 +0000 UTC m=+22.759713250" May 28 20:52:15.161376 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.161326 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xmjvg" podStartSLOduration=3.689087879 podStartE2EDuration="22.16131038s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.61332235 +0000 UTC m=+3.229157048" lastFinishedPulling="2026-05-28 20:52:14.085544849 +0000 UTC m=+21.701379549" observedRunningTime="2026-05-28 20:52:15.16086064 +0000 UTC m=+22.776695362" watchObservedRunningTime="2026-05-28 20:52:15.16131038 +0000 UTC m=+22.777145134" May 28 20:52:15.174953 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.174913 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gtpfl" podStartSLOduration=3.7389175100000003 podStartE2EDuration="22.174900194s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.604673343 +0000 UTC m=+3.220508047" lastFinishedPulling="2026-05-28 20:52:14.040656027 +0000 UTC m=+21.656490731" observedRunningTime="2026-05-28 20:52:15.174408118 +0000 UTC m=+22.790242837" watchObservedRunningTime="2026-05-28 20:52:15.174900194 +0000 UTC m=+22.790734913" May 28 20:52:15.191205 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.191165 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hzr92" podStartSLOduration=3.787819653 podStartE2EDuration="22.191152012s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.613089251 +0000 UTC m=+3.228923950" lastFinishedPulling="2026-05-28 20:52:14.016421606 +0000 UTC m=+21.632256309" observedRunningTime="2026-05-28 20:52:15.190572245 +0000 UTC m=+22.806406966" watchObservedRunningTime="2026-05-28 20:52:15.191152012 +0000 UTC m=+22.806986731" May 28 20:52:15.236549 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.236502 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-jstj6" podStartSLOduration=3.756056101 podStartE2EDuration="22.236488034s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.60682068 +0000 UTC m=+3.222655393" lastFinishedPulling="2026-05-28 20:52:14.087252626 +0000 UTC m=+21.703087326" observedRunningTime="2026-05-28 20:52:15.236010904 +0000 UTC m=+22.851845624" watchObservedRunningTime="2026-05-28 20:52:15.236488034 +0000 UTC m=+22.852322745" May 28 20:52:15.887625 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.887596 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 28 20:52:15.960182 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.960081 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T20:52:15.887620156Z","UUID":"4fddf880-34ba-4600-8571-4028bef9ce41","Handler":null,"Name":"","Endpoint":""} May 28 20:52:15.961840 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.961812 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 28 20:52:15.961840 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:15.961843 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 28 20:52:16.095021 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:16.094933 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9fph" event={"ID":"430ffa20-7a97-44d5-b166-bf23ed6c48f3","Type":"ContainerStarted","Data":"db3bda2041b94e366eb2922435d6371c531832e51ac9d80c29d4862df0afd49a"} May 28 20:52:16.097908 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:16.097873 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"8534fda928afd63f04deafa33d4ab2e3dc5eb2e4e23ad7f1023d76374bb391f4"} May 28 20:52:16.099621 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:16.099589 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" event={"ID":"bf6f9d86-22a2-4662-860d-fc00e48372a6","Type":"ContainerStarted","Data":"12fa3add04ff09251740441107b44d52c7f3a6f234befaf0f1716939eb28a27b"} May 28 20:52:16.987398 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:16.987370 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:16.987550 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:16.987405 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:16.987550 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:16.987503 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:16.987668 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:16.987619 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:17.103569 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:17.103521 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" event={"ID":"bf6f9d86-22a2-4662-860d-fc00e48372a6","Type":"ContainerStarted","Data":"8052b5091224ffe1f3e9770db025fb95894abb6b59aa276906641f6c8840b2a1"} May 28 20:52:17.121078 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:17.121031 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m9fph" podStartSLOduration=5.632539931 podStartE2EDuration="24.121017447s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.602741519 +0000 UTC m=+3.218576217" lastFinishedPulling="2026-05-28 20:52:14.091219022 +0000 UTC m=+21.707053733" observedRunningTime="2026-05-28 20:52:16.113389754 +0000 UTC m=+23.729224475" watchObservedRunningTime="2026-05-28 20:52:17.121017447 +0000 UTC m=+24.736852167" May 28 20:52:17.121282 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:17.121262 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xrb7r" podStartSLOduration=2.831568019 podStartE2EDuration="24.121257689s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.603410507 +0000 UTC m=+3.219245205" lastFinishedPulling="2026-05-28 20:52:16.893100174 +0000 UTC m=+24.508934875" observedRunningTime="2026-05-28 20:52:17.120941597 +0000 UTC m=+24.736776327" watchObservedRunningTime="2026-05-28 20:52:17.121257689 +0000 UTC m=+24.737092409" May 28 20:52:18.108031 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:18.107991 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"23f6888181133e14103a4d0907261066e93d97d7185b381f13797301ecefcf44"} May 28 20:52:18.895828 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:18.895795 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:52:18.896435 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:18.896413 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:52:18.987177 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:18.987147 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:18.987351 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:18.987262 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:18.987460 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:18.987147 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:18.987607 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:18.987573 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:19.109643 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:19.109612 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:52:19.110115 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:19.110047 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gtpfl" May 28 20:52:20.113794 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.113763 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" event={"ID":"8c8228ec-5064-45e2-bee8-aaa03fe0691c","Type":"ContainerStarted","Data":"b5567dc7c720ca28bea47ee403dfd73f766cd7a84b6dc2747a19506107a1a736"} May 28 20:52:20.114611 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.114030 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:20.114611 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.114054 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:20.129141 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.129115 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:20.129308 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.129297 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:20.216923 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.216748 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" podStartSLOduration=8.330952204999999 podStartE2EDuration="27.216735047s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.612047195 +0000 UTC m=+3.227881905" lastFinishedPulling="2026-05-28 20:52:14.497830042 +0000 UTC m=+22.113664747" observedRunningTime="2026-05-28 20:52:20.155192596 +0000 UTC m=+27.771027315" watchObservedRunningTime="2026-05-28 20:52:20.216735047 +0000 UTC m=+27.832569766" May 28 20:52:20.987462 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.987426 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:20.987662 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:20.987426 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:20.987662 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:20.987574 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:20.987784 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:20.987714 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:21.115224 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.115193 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" May 28 20:52:21.267813 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.267780 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mspmh"] May 28 20:52:21.267982 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.267897 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:21.268045 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:21.268023 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:21.270585 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.270557 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nh864"] May 28 20:52:21.270717 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.270643 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:21.270773 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:21.270744 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:21.298628 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:21.298603 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:22.987953 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:22.987915 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:22.987953 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:22.987953 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:22.988467 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:22.988034 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:22.988467 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:22.988140 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:24.122619 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:24.122527 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="d6e6ddd2cd63276d86dc238b1d7839ed9dcfe0e2a59913f9292a921383515ea6" exitCode=0 May 28 20:52:24.122619 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:24.122588 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"d6e6ddd2cd63276d86dc238b1d7839ed9dcfe0e2a59913f9292a921383515ea6"} May 28 20:52:24.986769 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:24.986742 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:24.986934 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:24.986746 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:24.986934 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:24.986841 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:24.986934 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:24.986913 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:26.127932 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.127731 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="6ede1f211608cf5204c506d8c91a47971f856497929e3b8724a96027e6f86958" exitCode=0 May 28 20:52:26.128305 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.127804 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"6ede1f211608cf5204c506d8c91a47971f856497929e3b8724a96027e6f86958"} May 28 20:52:26.642762 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.642730 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:26.642989 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.642877 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:26.642989 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.642935 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:58.642920233 +0000 UTC m=+66.258754935 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : object "openshift-multus"/"metrics-daemon-secret" not registered May 28 20:52:26.845201 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.845168 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:26.845333 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.845294 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 28 20:52:26.845333 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.845306 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 28 20:52:26.845333 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.845315 2570 projected.go:194] Error preparing data for projected volume kube-api-access-949ck for pod openshift-network-diagnostics/network-check-target-nh864: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:26.845426 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.845359 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck podName:7a95e461-88f6-4aee-af7b-4273b5da46e6 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:58.845346882 +0000 UTC m=+66.461181580 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-949ck" (UniqueName: "kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck") pod "network-check-target-nh864" (UID: "7a95e461-88f6-4aee-af7b-4273b5da46e6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 28 20:52:26.986848 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.986782 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:26.986848 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:26.986814 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:26.987005 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.986901 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:52:26.987040 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:26.986998 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nh864" podUID="7a95e461-88f6-4aee-af7b-4273b5da46e6" May 28 20:52:27.216782 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.216762 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-136-77.ec2.internal" event="NodeReady" May 28 20:52:27.217139 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.216873 2570 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 28 20:52:27.289299 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.289274 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:52:27.309103 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.309084 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.311810 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.311788 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 28 20:52:27.311915 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.311827 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 28 20:52:27.312266 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.312250 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 28 20:52:27.312498 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.312467 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrtjt\"" May 28 20:52:27.317125 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.317107 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:52:27.321946 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.321744 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 28 20:52:27.322694 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.322675 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2nlt4"] May 28 20:52:27.344416 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.344082 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vqhmn"] May 28 20:52:27.344615 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.344594 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.346504 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.346485 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 28 20:52:27.346596 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.346536 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6mfq4\"" May 28 20:52:27.346596 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.346546 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 28 20:52:27.358017 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.357999 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vqhmn"] May 28 20:52:27.358092 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.358023 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2nlt4"] May 28 20:52:27.358129 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.358112 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.361579 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.361562 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 28 20:52:27.361930 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.361917 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 28 20:52:27.362008 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.361960 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gngfj\"" May 28 20:52:27.362294 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.362280 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 28 20:52:27.450226 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450194 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk7vs\" (UniqueName: \"kubernetes.io/projected/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-kube-api-access-jk7vs\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.450377 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450233 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450377 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450293 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m97l4\" (UniqueName: \"kubernetes.io/projected/9b62715f-a93e-4544-8a69-3d443630e2d2-kube-api-access-m97l4\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.450377 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450316 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450500 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450376 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450500 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-tmp-dir\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.450500 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450500 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450457 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.450618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450498 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4nlf\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.450618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-config-volume\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.450762 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.450618 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.551760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551683 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.551760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551719 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-tmp-dir\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.551760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551740 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.551760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551760 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.552050 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551795 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552050 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551824 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552050 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551855 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552050 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.551873 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:27.552050 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.551876 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r4nlf\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552237 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.551980 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:28.051956914 +0000 UTC m=+35.667791633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:27.552415 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552379 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-config-volume\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.552513 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552460 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.552513 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-tmp-dir\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.552624 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552525 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jk7vs\" (UniqueName: \"kubernetes.io/projected/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-kube-api-access-jk7vs\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.552624 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552577 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552713 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552646 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m97l4\" (UniqueName: \"kubernetes.io/projected/9b62715f-a93e-4544-8a69-3d443630e2d2-kube-api-access-m97l4\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.552713 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.552696 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.552886 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.552868 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:27.552954 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.552892 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:27.553339 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.553298 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.553440 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.553384 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:28.053349468 +0000 UTC m=+35.669184179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:27.554510 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.553960 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:27.554510 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:27.554021 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:28.054006389 +0000 UTC m=+35.669841103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:27.554666 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.554540 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.554666 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.554637 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.557275 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.557256 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.557383 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.557259 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.563392 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.563367 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m97l4\" (UniqueName: \"kubernetes.io/projected/9b62715f-a93e-4544-8a69-3d443630e2d2-kube-api-access-m97l4\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:27.563832 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.563814 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.563963 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.563946 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4nlf\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:27.564700 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.564682 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-config-volume\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:27.566159 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:27.566141 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk7vs\" (UniqueName: \"kubernetes.io/projected/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-kube-api-access-jk7vs\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:28.056428 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.056398 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:28.056609 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.056448 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:28.056609 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056549 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:28.056609 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056563 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:28.056609 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056549 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:28.056609 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.056600 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:28.056772 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056633 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:29.05661578 +0000 UTC m=+36.672450479 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:28.056772 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056659 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:28.056772 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056671 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:29.056659161 +0000 UTC m=+36.672493863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:28.056772 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:28.056692 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:29.05668321 +0000 UTC m=+36.672517908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:28.133670 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.133645 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="f4da8f1ff8fda8d9f8d5df01f7c1d8accd0523a601c76712b40a15dd2c7a020d" exitCode=0 May 28 20:52:28.133808 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.133693 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"f4da8f1ff8fda8d9f8d5df01f7c1d8accd0523a601c76712b40a15dd2c7a020d"} May 28 20:52:28.990113 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.990084 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:28.990662 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.990090 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:28.998334 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.998313 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 20:52:28.998448 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.998412 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 20:52:28.998448 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.998412 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 20:52:28.999088 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.999070 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6jf7t\"" May 28 20:52:28.999167 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:28.999150 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbvdl\"" May 28 20:52:29.064301 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:29.064267 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:29.064521 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:29.064326 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:29.064521 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064439 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:29.064521 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064452 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:29.064521 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:29.064484 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:29.064521 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064521 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:31.064501646 +0000 UTC m=+38.680336346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:29.064788 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064440 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:29.064788 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064626 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:31.064606695 +0000 UTC m=+38.680441410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:29.064788 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064555 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:29.064788 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:29.064671 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:31.064660084 +0000 UTC m=+38.680494805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:30.725430 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.725394 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb"] May 28 20:52:30.741662 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.741637 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx"] May 28 20:52:30.741836 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.741803 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.746185 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.746153 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 28 20:52:30.746185 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.746171 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 28 20:52:30.746455 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.746230 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 28 20:52:30.746455 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.746279 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-n6p6r\"" May 28 20:52:30.746608 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.746495 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 28 20:52:30.752178 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.752154 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt"] May 28 20:52:30.752334 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.752320 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.755430 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.755077 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 28 20:52:30.755430 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.755082 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 28 20:52:30.755609 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.755564 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 28 20:52:30.763044 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.763019 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt"] May 28 20:52:30.763044 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.763051 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb"] May 28 20:52:30.764303 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.764279 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.764655 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.764635 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 28 20:52:30.768623 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.767456 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx"] May 28 20:52:30.769262 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.769242 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" May 28 20:52:30.877575 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.877718 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877587 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.877718 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ac5e2436-5635-4a10-bccc-b5c60f522e22-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.877832 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84vm6\" (UniqueName: \"kubernetes.io/projected/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-kube-api-access-84vm6\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.877832 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.877933 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877849 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.877933 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgrht\" (UniqueName: \"kubernetes.io/projected/ac5e2436-5635-4a10-bccc-b5c60f522e22-kube-api-access-zgrht\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.878019 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b536741a-83f3-4f46-8d38-05624e84706b-tmp\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.878019 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877968 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.878019 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.877995 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b536741a-83f3-4f46-8d38-05624e84706b-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.878140 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.878021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzhxd\" (UniqueName: \"kubernetes.io/projected/b536741a-83f3-4f46-8d38-05624e84706b-kube-api-access-qzhxd\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.979058 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.978981 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979058 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979020 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.979058 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979044 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ac5e2436-5635-4a10-bccc-b5c60f522e22-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979084 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-84vm6\" (UniqueName: \"kubernetes.io/projected/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-kube-api-access-84vm6\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979112 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979133 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979168 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zgrht\" (UniqueName: \"kubernetes.io/projected/ac5e2436-5635-4a10-bccc-b5c60f522e22-kube-api-access-zgrht\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979193 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b536741a-83f3-4f46-8d38-05624e84706b-tmp\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979219 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979250 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b536741a-83f3-4f46-8d38-05624e84706b-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.979316 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979286 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qzhxd\" (UniqueName: \"kubernetes.io/projected/b536741a-83f3-4f46-8d38-05624e84706b-kube-api-access-qzhxd\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.980171 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.979884 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/ac5e2436-5635-4a10-bccc-b5c60f522e22-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.980383 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.980359 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b536741a-83f3-4f46-8d38-05624e84706b-tmp\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.982302 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.982256 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:30.982414 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.982349 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-ca\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.982794 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.982767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.982874 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.982802 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-hub\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:30.983160 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.983139 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/b536741a-83f3-4f46-8d38-05624e84706b-klusterlet-config\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:30.983724 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:30.983703 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/ac5e2436-5635-4a10-bccc-b5c60f522e22-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:31.010085 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.010055 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgrht\" (UniqueName: \"kubernetes.io/projected/ac5e2436-5635-4a10-bccc-b5c60f522e22-kube-api-access-zgrht\") pod \"cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx\" (UID: \"ac5e2436-5635-4a10-bccc-b5c60f522e22\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:31.016977 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.016949 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzhxd\" (UniqueName: \"kubernetes.io/projected/b536741a-83f3-4f46-8d38-05624e84706b-kube-api-access-qzhxd\") pod \"klusterlet-addon-workmgr-67c9b47b76-zhhrt\" (UID: \"b536741a-83f3-4f46-8d38-05624e84706b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:31.018983 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.018958 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84vm6\" (UniqueName: \"kubernetes.io/projected/a50cfece-6c6d-42ee-b778-5a9ee0ba024e-kube-api-access-84vm6\") pod \"managed-serviceaccount-addon-agent-77b6565946-xqkmb\" (UID: \"a50cfece-6c6d-42ee-b778-5a9ee0ba024e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:31.055859 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.055827 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-rqdfl"] May 28 20:52:31.060830 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.060802 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" May 28 20:52:31.074064 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.074040 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:52:31.080360 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.080335 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:31.080500 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.080428 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:31.080500 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080493 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.080506 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080550 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:35.080533758 +0000 UTC m=+42.696368474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.080595 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqdfl"] May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080616 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080631 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080639 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080678 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:35.080663878 +0000 UTC m=+42.696498575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:31.080696 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:35.080686881 +0000 UTC m=+42.696521620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:31.080806 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.080718 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.084036 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.084010 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 28 20:52:31.090034 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.090017 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:31.181712 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.181201 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-kubelet-config\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.182202 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.181982 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-dbus\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.182202 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.182023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5746890a-97c1-4544-971b-e0d15add347d-original-pull-secret\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.283236 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.283203 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-kubelet-config\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.283494 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.283274 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-dbus\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.283494 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.283297 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5746890a-97c1-4544-971b-e0d15add347d-original-pull-secret\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.283494 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.283330 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-kubelet-config\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.283689 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.283638 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5746890a-97c1-4544-971b-e0d15add347d-dbus\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.287041 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.286999 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5746890a-97c1-4544-971b-e0d15add347d-original-pull-secret\") pod \"global-pull-secret-syncer-rqdfl\" (UID: \"5746890a-97c1-4544-971b-e0d15add347d\") " pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.304154 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.304129 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt"] May 28 20:52:31.310983 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.310942 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb"] May 28 20:52:31.311430 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:52:31.311406 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb536741a_83f3_4f46_8d38_05624e84706b.slice/crio-ebad042c6b672454fa02e456a38686bb06ff90555f6b9362dfc63a17f447e6d0 WatchSource:0}: Error finding container ebad042c6b672454fa02e456a38686bb06ff90555f6b9362dfc63a17f447e6d0: Status 404 returned error can't find the container with id ebad042c6b672454fa02e456a38686bb06ff90555f6b9362dfc63a17f447e6d0 May 28 20:52:31.314582 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:52:31.314558 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda50cfece_6c6d_42ee_b778_5a9ee0ba024e.slice/crio-7710d7b4755db2082efee6d8d4034a0c105661d94f1474356d1f4e0fd6041241 WatchSource:0}: Error finding container 7710d7b4755db2082efee6d8d4034a0c105661d94f1474356d1f4e0fd6041241: Status 404 returned error can't find the container with id 7710d7b4755db2082efee6d8d4034a0c105661d94f1474356d1f4e0fd6041241 May 28 20:52:31.316174 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.316152 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx"] May 28 20:52:31.325664 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:52:31.325640 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac5e2436_5635_4a10_bccc_b5c60f522e22.slice/crio-0e8eff01d0d391ec0a7495f34ee50bf959007ee1184fd0eec5be7f493c42b3c6 WatchSource:0}: Error finding container 0e8eff01d0d391ec0a7495f34ee50bf959007ee1184fd0eec5be7f493c42b3c6: Status 404 returned error can't find the container with id 0e8eff01d0d391ec0a7495f34ee50bf959007ee1184fd0eec5be7f493c42b3c6 May 28 20:52:31.393083 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.393049 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqdfl" May 28 20:52:31.542837 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:31.542749 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqdfl"] May 28 20:52:31.557074 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:52:31.557044 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5746890a_97c1_4544_971b_e0d15add347d.slice/crio-8c8143da3dc3451449398f7e98568f4d726989007a881059f4042f582c4225df WatchSource:0}: Error finding container 8c8143da3dc3451449398f7e98568f4d726989007a881059f4042f582c4225df: Status 404 returned error can't find the container with id 8c8143da3dc3451449398f7e98568f4d726989007a881059f4042f582c4225df May 28 20:52:32.143348 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:32.143310 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerStarted","Data":"0e8eff01d0d391ec0a7495f34ee50bf959007ee1184fd0eec5be7f493c42b3c6"} May 28 20:52:32.144805 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:32.144747 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" event={"ID":"a50cfece-6c6d-42ee-b778-5a9ee0ba024e","Type":"ContainerStarted","Data":"7710d7b4755db2082efee6d8d4034a0c105661d94f1474356d1f4e0fd6041241"} May 28 20:52:32.145949 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:32.145916 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqdfl" event={"ID":"5746890a-97c1-4544-971b-e0d15add347d","Type":"ContainerStarted","Data":"8c8143da3dc3451449398f7e98568f4d726989007a881059f4042f582c4225df"} May 28 20:52:32.147588 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:32.147563 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" event={"ID":"b536741a-83f3-4f46-8d38-05624e84706b","Type":"ContainerStarted","Data":"ebad042c6b672454fa02e456a38686bb06ff90555f6b9362dfc63a17f447e6d0"} May 28 20:52:35.117202 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:35.117159 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:35.117241 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:35.117292 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117315 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117378 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:43.117358992 +0000 UTC m=+50.733193701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117389 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117402 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117416 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117433 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:43.117422179 +0000 UTC m=+50.733256879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:35.117682 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:35.117500 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:43.117461886 +0000 UTC m=+50.733296588 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:43.171565 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.171520 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerStarted","Data":"c56309aa90fd01be7c2a11ef5b17684aeb90d2c13a5af2ecad84d01fbaf692c9"} May 28 20:52:43.172910 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.172882 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" event={"ID":"a50cfece-6c6d-42ee-b778-5a9ee0ba024e","Type":"ContainerStarted","Data":"d156ab2647b0653fa35836037a66bf1089fc1321d47da9cef9c21f47346fe077"} May 28 20:52:43.174419 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.174391 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqdfl" event={"ID":"5746890a-97c1-4544-971b-e0d15add347d","Type":"ContainerStarted","Data":"01690354e86fb99d0d70ea3a456bbe8c0dfe5d0fb0a2059f696a5f34f881d791"} May 28 20:52:43.175918 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.175891 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" event={"ID":"b536741a-83f3-4f46-8d38-05624e84706b","Type":"ContainerStarted","Data":"b086f53241987c8faceeb008df799a381c74957027d9f222667bd319bf30cb2e"} May 28 20:52:43.176246 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.176229 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:43.178079 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.178060 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" May 28 20:52:43.178863 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.178843 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="6d9e9fc480466465059b0f85d9614aee33a5cda1129c4c590ca8fd1c2a42ba38" exitCode=0 May 28 20:52:43.178945 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.178898 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"6d9e9fc480466465059b0f85d9614aee33a5cda1129c4c590ca8fd1c2a42ba38"} May 28 20:52:43.190040 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.190015 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:43.190183 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190169 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:43.190235 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190225 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:59.190205133 +0000 UTC m=+66.806039845 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:43.190372 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.190353 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:43.190442 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.190428 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:43.190585 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190565 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:43.190698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190612 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:59.190594105 +0000 UTC m=+66.806428819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:43.190820 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190806 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:43.190883 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190825 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:43.190883 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:43.190871 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:52:59.190855438 +0000 UTC m=+66.806690141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:43.200584 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.198808 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-77b6565946-xqkmb" podStartSLOduration=1.82373659 podStartE2EDuration="13.198795127s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:31.316761301 +0000 UTC m=+38.932596011" lastFinishedPulling="2026-05-28 20:52:42.691819837 +0000 UTC m=+50.307654548" observedRunningTime="2026-05-28 20:52:43.198045796 +0000 UTC m=+50.813880520" watchObservedRunningTime="2026-05-28 20:52:43.198795127 +0000 UTC m=+50.814629846" May 28 20:52:43.233047 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.232862 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67c9b47b76-zhhrt" podStartSLOduration=2.177256122 podStartE2EDuration="13.232846709s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:31.313613675 +0000 UTC m=+38.929448372" lastFinishedPulling="2026-05-28 20:52:42.369204259 +0000 UTC m=+49.985038959" observedRunningTime="2026-05-28 20:52:43.231737004 +0000 UTC m=+50.847571727" watchObservedRunningTime="2026-05-28 20:52:43.232846709 +0000 UTC m=+50.848681430" May 28 20:52:43.302087 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:43.301867 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rqdfl" podStartSLOduration=1.520998289 podStartE2EDuration="12.301848106s" podCreationTimestamp="2026-05-28 20:52:31 +0000 UTC" firstStartedPulling="2026-05-28 20:52:31.558846283 +0000 UTC m=+39.174680980" lastFinishedPulling="2026-05-28 20:52:42.3396961 +0000 UTC m=+49.955530797" observedRunningTime="2026-05-28 20:52:43.300758036 +0000 UTC m=+50.916592756" watchObservedRunningTime="2026-05-28 20:52:43.301848106 +0000 UTC m=+50.917682828" May 28 20:52:44.184264 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:44.184228 2570 generic.go:358] "Generic (PLEG): container finished" podID="2d2d247b-766d-4b6f-845f-0d4a22141dc7" containerID="92cd8de2846a9ecf81f99d6cf2033acee8b03d921470f71b2a4c31ccfa9e3472" exitCode=0 May 28 20:52:44.184769 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:44.184304 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerDied","Data":"92cd8de2846a9ecf81f99d6cf2033acee8b03d921470f71b2a4c31ccfa9e3472"} May 28 20:52:45.189734 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:45.189698 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p79nd" event={"ID":"2d2d247b-766d-4b6f-845f-0d4a22141dc7","Type":"ContainerStarted","Data":"b5f623410c62b2964403fda8ae217fbda82acd86dff83ca7ea09c89078fe2503"} May 28 20:52:45.191418 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:45.191393 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerStarted","Data":"027fd34d5ce57dbbc98ef77353a3eb9670825b4846b0f5240a0575c1af05b315"} May 28 20:52:45.191418 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:45.191419 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerStarted","Data":"cdc1d7eb4c9a1641df7e0fdd3886d69d93f97868fa50ddcd074aa2ce5c85d876"} May 28 20:52:45.239580 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:45.239530 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p79nd" podStartSLOduration=6.126537685 podStartE2EDuration="52.239516008s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:51:55.6089563 +0000 UTC m=+3.224790998" lastFinishedPulling="2026-05-28 20:52:41.721934622 +0000 UTC m=+49.337769321" observedRunningTime="2026-05-28 20:52:45.238991346 +0000 UTC m=+52.854826066" watchObservedRunningTime="2026-05-28 20:52:45.239516008 +0000 UTC m=+52.855350727" May 28 20:52:45.283194 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:45.283144 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" podStartSLOduration=2.272307917 podStartE2EDuration="15.283128888s" podCreationTimestamp="2026-05-28 20:52:30 +0000 UTC" firstStartedPulling="2026-05-28 20:52:31.327693381 +0000 UTC m=+38.943528088" lastFinishedPulling="2026-05-28 20:52:44.338514348 +0000 UTC m=+51.954349059" observedRunningTime="2026-05-28 20:52:45.282685046 +0000 UTC m=+52.898519767" watchObservedRunningTime="2026-05-28 20:52:45.283128888 +0000 UTC m=+52.898963608" May 28 20:52:53.130023 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:53.129995 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-j8294" May 28 20:52:58.708825 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.708785 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:52:58.711094 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.711073 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 28 20:52:58.719159 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:58.719142 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 20:52:58.719236 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:58.719192 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:02.719178587 +0000 UTC m=+130.335013285 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : secret "metrics-daemon-secret" not found May 28 20:52:58.910902 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.910860 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:58.913391 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.913373 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 28 20:52:58.923000 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.922977 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 28 20:52:58.934410 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:58.934381 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-949ck\" (UniqueName: \"kubernetes.io/projected/7a95e461-88f6-4aee-af7b-4273b5da46e6-kube-api-access-949ck\") pod \"network-check-target-nh864\" (UID: \"7a95e461-88f6-4aee-af7b-4273b5da46e6\") " pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:59.001700 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.001674 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6jf7t\"" May 28 20:52:59.009259 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.009236 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:52:59.133148 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.133115 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nh864"] May 28 20:52:59.137375 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:52:59.137344 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a95e461_88f6_4aee_af7b_4273b5da46e6.slice/crio-22262d4480a05ebbf293aa8eba0d083c9c4a71e69937b0f6fa5b115cbdcc579d WatchSource:0}: Error finding container 22262d4480a05ebbf293aa8eba0d083c9c4a71e69937b0f6fa5b115cbdcc579d: Status 404 returned error can't find the container with id 22262d4480a05ebbf293aa8eba0d083c9c4a71e69937b0f6fa5b115cbdcc579d May 28 20:52:59.213878 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.213850 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:52:59.214052 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.213897 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:52:59.214052 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.213931 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:52:59.214052 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214017 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:52:59.214052 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214023 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:52:59.214052 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214049 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:52:59.214266 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214069 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:31.214056126 +0000 UTC m=+98.829890823 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:52:59.214266 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214116 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:31.214102457 +0000 UTC m=+98.829937155 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:52:59.214266 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214027 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:52:59.214266 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:52:59.214175 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:53:31.214159774 +0000 UTC m=+98.829994486 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:52:59.228580 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:52:59.228555 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nh864" event={"ID":"7a95e461-88f6-4aee-af7b-4273b5da46e6","Type":"ContainerStarted","Data":"22262d4480a05ebbf293aa8eba0d083c9c4a71e69937b0f6fa5b115cbdcc579d"} May 28 20:53:03.239364 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:03.239340 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nh864" event={"ID":"7a95e461-88f6-4aee-af7b-4273b5da46e6","Type":"ContainerStarted","Data":"150b52fb41735b9ff56205f20582230de6054300823fa0e4f34f3acc31e5d336"} May 28 20:53:04.241257 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:04.241222 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:53:31.260141 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:31.260100 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:31.260162 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:31.260200 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260259 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260297 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260344 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls podName:fdbfb640-0d41-45f2-9aa8-e3bf40592de5 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:35.260322719 +0000 UTC m=+162.876157494 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls") pod "dns-default-2nlt4" (UID: "fdbfb640-0d41-45f2-9aa8-e3bf40592de5") : secret "dns-default-metrics-tls" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260342 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260366 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-776cd9fc89-d9h22: secret "image-registry-tls" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260360 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert podName:9b62715f-a93e-4544-8a69-3d443630e2d2 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:35.260353499 +0000 UTC m=+162.876188201 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert") pod "ingress-canary-vqhmn" (UID: "9b62715f-a93e-4544-8a69-3d443630e2d2") : secret "canary-serving-cert" not found May 28 20:53:31.260698 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:53:31.260420 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls podName:19b0e0c8-8889-49b2-babe-2762d09ad773 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:35.260408738 +0000 UTC m=+162.876243441 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls") pod "image-registry-776cd9fc89-d9h22" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773") : secret "image-registry-tls" not found May 28 20:53:35.246823 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:35.246793 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nh864" May 28 20:53:35.261362 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:35.261315 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nh864" podStartSLOduration=98.214231641 podStartE2EDuration="1m42.26130169s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:52:59.139227642 +0000 UTC m=+66.755062347" lastFinishedPulling="2026-05-28 20:53:03.186297697 +0000 UTC m=+70.802132396" observedRunningTime="2026-05-28 20:53:04.262279152 +0000 UTC m=+71.878113884" watchObservedRunningTime="2026-05-28 20:53:35.26130169 +0000 UTC m=+102.877136442" May 28 20:53:59.182901 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.182869 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd"] May 28 20:53:59.185763 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.185747 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" May 28 20:53:59.187861 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.187843 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-crs47\"" May 28 20:53:59.195922 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.195900 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd"] May 28 20:53:59.263960 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.263931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfqrw\" (UniqueName: \"kubernetes.io/projected/317edb43-aec7-408d-8489-237cf2af8a1b-kube-api-access-kfqrw\") pod \"network-check-source-7f44b6dd87-zm2vd\" (UID: \"317edb43-aec7-408d-8489-237cf2af8a1b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" May 28 20:53:59.364623 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.364590 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kfqrw\" (UniqueName: \"kubernetes.io/projected/317edb43-aec7-408d-8489-237cf2af8a1b-kube-api-access-kfqrw\") pod \"network-check-source-7f44b6dd87-zm2vd\" (UID: \"317edb43-aec7-408d-8489-237cf2af8a1b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" May 28 20:53:59.373147 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.373115 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfqrw\" (UniqueName: \"kubernetes.io/projected/317edb43-aec7-408d-8489-237cf2af8a1b-kube-api-access-kfqrw\") pod \"network-check-source-7f44b6dd87-zm2vd\" (UID: \"317edb43-aec7-408d-8489-237cf2af8a1b\") " pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" May 28 20:53:59.494001 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.493927 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" May 28 20:53:59.608171 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:53:59.608142 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd"] May 28 20:53:59.611436 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:53:59.611408 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod317edb43_aec7_408d_8489_237cf2af8a1b.slice/crio-eac5c2e620085c0bd4a40f51d8286366f605a1fe3fc4df23ab7a1e334bc083a2 WatchSource:0}: Error finding container eac5c2e620085c0bd4a40f51d8286366f605a1fe3fc4df23ab7a1e334bc083a2: Status 404 returned error can't find the container with id eac5c2e620085c0bd4a40f51d8286366f605a1fe3fc4df23ab7a1e334bc083a2 May 28 20:54:00.370076 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:00.370035 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" event={"ID":"317edb43-aec7-408d-8489-237cf2af8a1b","Type":"ContainerStarted","Data":"662e7bd2dc2a35a191c907377e1718d9cd9e828cc2a3928e10f0479613132377"} May 28 20:54:00.370076 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:00.370079 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" event={"ID":"317edb43-aec7-408d-8489-237cf2af8a1b","Type":"ContainerStarted","Data":"eac5c2e620085c0bd4a40f51d8286366f605a1fe3fc4df23ab7a1e334bc083a2"} May 28 20:54:00.384841 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:00.384794 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7f44b6dd87-zm2vd" podStartSLOduration=1.384778298 podStartE2EDuration="1.384778298s" podCreationTimestamp="2026-05-28 20:53:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:54:00.3839067 +0000 UTC m=+127.999741433" watchObservedRunningTime="2026-05-28 20:54:00.384778298 +0000 UTC m=+128.000613018" May 28 20:54:02.793890 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:02.793844 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:54:02.794298 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:02.793992 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 28 20:54:02.794298 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:02.794058 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs podName:df288737-cf68-4b61-9f6b-4a87f5df1a98 nodeName:}" failed. No retries permitted until 2026-05-28 20:56:04.794043301 +0000 UTC m=+252.409877999 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs") pod "network-metrics-daemon-mspmh" (UID: "df288737-cf68-4b61-9f6b-4a87f5df1a98") : secret "metrics-daemon-secret" not found May 28 20:54:04.869175 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:04.869144 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jstj6_1bed4256-2823-4ff3-a5e1-dabf5429faa7/dns-node-resolver/0.log" May 28 20:54:06.269543 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:06.269517 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hzr92_9d377526-7f51-4275-ad9e-c99a0a1766ed/node-ca/0.log" May 28 20:54:11.075884 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:11.075835 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" podUID="ac5e2436-5635-4a10-bccc-b5c60f522e22" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 20:54:21.075905 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:21.075868 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" podUID="ac5e2436-5635-4a10-bccc-b5c60f522e22" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 20:54:26.240214 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.240179 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gw2nm"] May 28 20:54:26.243322 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.243293 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.245572 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.245547 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 28 20:54:26.245688 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.245576 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 28 20:54:26.245688 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.245550 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 28 20:54:26.245688 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.245627 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gc8h7\"" May 28 20:54:26.245688 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.245579 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 28 20:54:26.254133 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.254112 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gw2nm"] May 28 20:54:26.368145 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.368109 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-crio-socket\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.368145 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.368145 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztwxv\" (UniqueName: \"kubernetes.io/projected/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-api-access-ztwxv\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.368353 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.368186 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-data-volume\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.368353 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.368233 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.368353 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.368256 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469084 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469053 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-crio-socket\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469084 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469084 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ztwxv\" (UniqueName: \"kubernetes.io/projected/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-api-access-ztwxv\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469280 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469171 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-crio-socket\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469280 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469207 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-data-volume\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469280 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469268 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469377 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469306 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469599 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469580 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-data-volume\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.469775 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.469761 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.471535 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.471509 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.476302 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.476279 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztwxv\" (UniqueName: \"kubernetes.io/projected/0f0a4eac-7264-4a7b-9a79-84a9c507ffd7-kube-api-access-ztwxv\") pod \"insights-runtime-extractor-gw2nm\" (UID: \"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7\") " pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.551707 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.551678 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gw2nm" May 28 20:54:26.675149 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:26.675119 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gw2nm"] May 28 20:54:26.678448 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:54:26.678421 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f0a4eac_7264_4a7b_9a79_84a9c507ffd7.slice/crio-29a18966892b951a236c0aa17b18e2af17e563c00244342c4087fec80e43703d WatchSource:0}: Error finding container 29a18966892b951a236c0aa17b18e2af17e563c00244342c4087fec80e43703d: Status 404 returned error can't find the container with id 29a18966892b951a236c0aa17b18e2af17e563c00244342c4087fec80e43703d May 28 20:54:27.429148 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:27.429117 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gw2nm" event={"ID":"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7","Type":"ContainerStarted","Data":"1aac1aaed23da23341300e43e079d223e854f43f3ad9918625b5ca0aeaca4641"} May 28 20:54:27.429414 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:27.429166 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gw2nm" event={"ID":"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7","Type":"ContainerStarted","Data":"29a18966892b951a236c0aa17b18e2af17e563c00244342c4087fec80e43703d"} May 28 20:54:28.433735 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:28.433690 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gw2nm" event={"ID":"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7","Type":"ContainerStarted","Data":"16b3f5db547b4e48a8d0bb0a52ed1a1f6fd1fb5240d2f679269f42ed17e9dca7"} May 28 20:54:29.437366 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:29.437322 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gw2nm" event={"ID":"0f0a4eac-7264-4a7b-9a79-84a9c507ffd7","Type":"ContainerStarted","Data":"546207df2da9edfa810cbae8416293276a46397eee8592edaaaa99a27ffaf80f"} May 28 20:54:29.453364 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:29.453274 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gw2nm" podStartSLOduration=1.295506794 podStartE2EDuration="3.453258483s" podCreationTimestamp="2026-05-28 20:54:26 +0000 UTC" firstStartedPulling="2026-05-28 20:54:26.737058123 +0000 UTC m=+154.352892822" lastFinishedPulling="2026-05-28 20:54:28.894809813 +0000 UTC m=+156.510644511" observedRunningTime="2026-05-28 20:54:29.452582108 +0000 UTC m=+157.068416828" watchObservedRunningTime="2026-05-28 20:54:29.453258483 +0000 UTC m=+157.069093205" May 28 20:54:30.385951 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:30.385885 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" May 28 20:54:30.391039 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:30.391008 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-2nlt4" podUID="fdbfb640-0d41-45f2-9aa8-e3bf40592de5" May 28 20:54:30.396184 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:30.396160 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-vqhmn" podUID="9b62715f-a93e-4544-8a69-3d443630e2d2" May 28 20:54:30.440232 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:30.440205 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2nlt4" May 28 20:54:30.440638 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:30.440317 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:54:31.075895 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.075818 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" podUID="ac5e2436-5635-4a10-bccc-b5c60f522e22" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 28 20:54:31.075895 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.075891 2570 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" May 28 20:54:31.076446 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.076417 2570 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"027fd34d5ce57dbbc98ef77353a3eb9670825b4846b0f5240a0575c1af05b315"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 28 20:54:31.076521 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.076506 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" podUID="ac5e2436-5635-4a10-bccc-b5c60f522e22" containerName="service-proxy" containerID="cri-o://027fd34d5ce57dbbc98ef77353a3eb9670825b4846b0f5240a0575c1af05b315" gracePeriod=30 May 28 20:54:31.444930 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.444847 2570 generic.go:358] "Generic (PLEG): container finished" podID="ac5e2436-5635-4a10-bccc-b5c60f522e22" containerID="027fd34d5ce57dbbc98ef77353a3eb9670825b4846b0f5240a0575c1af05b315" exitCode=2 May 28 20:54:31.444930 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.444895 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerDied","Data":"027fd34d5ce57dbbc98ef77353a3eb9670825b4846b0f5240a0575c1af05b315"} May 28 20:54:31.444930 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:31.444925 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5dcd6bbd65-c2sgx" event={"ID":"ac5e2436-5635-4a10-bccc-b5c60f522e22","Type":"ContainerStarted","Data":"5b05afb2eeb5b408473116d9674a4abe8e4a089327bdbf1aa92c5d46e8ebdfd6"} May 28 20:54:32.005316 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:32.005261 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mspmh" podUID="df288737-cf68-4b61-9f6b-4a87f5df1a98" May 28 20:54:35.335402 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.335361 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:54:35.335402 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.335409 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:54:35.335857 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.335452 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:54:35.337706 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.337680 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdbfb640-0d41-45f2-9aa8-e3bf40592de5-metrics-tls\") pod \"dns-default-2nlt4\" (UID: \"fdbfb640-0d41-45f2-9aa8-e3bf40592de5\") " pod="openshift-dns/dns-default-2nlt4" May 28 20:54:35.337817 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.337750 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"image-registry-776cd9fc89-d9h22\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:54:35.337817 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.337795 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b62715f-a93e-4544-8a69-3d443630e2d2-cert\") pod \"ingress-canary-vqhmn\" (UID: \"9b62715f-a93e-4544-8a69-3d443630e2d2\") " pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:54:35.543832 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.543802 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6mfq4\"" May 28 20:54:35.543965 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.543845 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jrtjt\"" May 28 20:54:35.551417 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.551385 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:54:35.551417 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.551411 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2nlt4" May 28 20:54:35.680235 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.680204 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2nlt4"] May 28 20:54:35.683151 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:54:35.683119 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbfb640_0d41_45f2_9aa8_e3bf40592de5.slice/crio-1e8a446d30cf95693794f908f2620627c26925a73c88134e8e337e6e3a5399a9 WatchSource:0}: Error finding container 1e8a446d30cf95693794f908f2620627c26925a73c88134e8e337e6e3a5399a9: Status 404 returned error can't find the container with id 1e8a446d30cf95693794f908f2620627c26925a73c88134e8e337e6e3a5399a9 May 28 20:54:35.695251 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:35.695230 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:54:35.697875 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:54:35.697853 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b0e0c8_8889_49b2_babe_2762d09ad773.slice/crio-6e57426ec36b37f3d0c44261e1814326756467712bbd7010f878826318631362 WatchSource:0}: Error finding container 6e57426ec36b37f3d0c44261e1814326756467712bbd7010f878826318631362: Status 404 returned error can't find the container with id 6e57426ec36b37f3d0c44261e1814326756467712bbd7010f878826318631362 May 28 20:54:36.459635 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:36.459593 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" event={"ID":"19b0e0c8-8889-49b2-babe-2762d09ad773","Type":"ContainerStarted","Data":"b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634"} May 28 20:54:36.460187 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:36.459641 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" event={"ID":"19b0e0c8-8889-49b2-babe-2762d09ad773","Type":"ContainerStarted","Data":"6e57426ec36b37f3d0c44261e1814326756467712bbd7010f878826318631362"} May 28 20:54:36.460187 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:36.459712 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:54:36.460919 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:36.460892 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2nlt4" event={"ID":"fdbfb640-0d41-45f2-9aa8-e3bf40592de5","Type":"ContainerStarted","Data":"1e8a446d30cf95693794f908f2620627c26925a73c88134e8e337e6e3a5399a9"} May 28 20:54:36.477810 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:36.477625 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" podStartSLOduration=163.47760963 podStartE2EDuration="2m43.47760963s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 20:54:36.476840907 +0000 UTC m=+164.092675628" watchObservedRunningTime="2026-05-28 20:54:36.47760963 +0000 UTC m=+164.093444350" May 28 20:54:37.465041 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:37.464937 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2nlt4" event={"ID":"fdbfb640-0d41-45f2-9aa8-e3bf40592de5","Type":"ContainerStarted","Data":"b4172ee461dd1a730e2571f73807e6cea26c3efd89fbbac3aa4fd8dbfb764e8f"} May 28 20:54:37.465041 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:37.465002 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2nlt4" event={"ID":"fdbfb640-0d41-45f2-9aa8-e3bf40592de5","Type":"ContainerStarted","Data":"9a6e78b3478b78e372a295dafdb76054fa3d82ee2e0848b98a50e2ea8ec901a6"} May 28 20:54:37.479972 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:37.479925 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-2nlt4" podStartSLOduration=129.061166245 podStartE2EDuration="2m10.479907409s" podCreationTimestamp="2026-05-28 20:52:27 +0000 UTC" firstStartedPulling="2026-05-28 20:54:35.684975138 +0000 UTC m=+163.300809847" lastFinishedPulling="2026-05-28 20:54:37.103716298 +0000 UTC m=+164.719551011" observedRunningTime="2026-05-28 20:54:37.479419875 +0000 UTC m=+165.095254633" watchObservedRunningTime="2026-05-28 20:54:37.479907409 +0000 UTC m=+165.095742129" May 28 20:54:38.468037 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:38.468007 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2nlt4" May 28 20:54:41.637063 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.637030 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n2jbk"] May 28 20:54:41.640094 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.640078 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.641998 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.641972 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 28 20:54:41.642131 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642056 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 28 20:54:41.642131 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642065 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 28 20:54:41.642131 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642089 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 28 20:54:41.642539 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642520 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 28 20:54:41.642662 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642544 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pxmvg\"" May 28 20:54:41.642725 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.642712 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 28 20:54:41.787576 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787531 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-textfile\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787576 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787580 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-metrics-client-ca\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp8td\" (UniqueName: \"kubernetes.io/projected/f1985411-2d30-4df7-9e4d-0208bf97dcf4-kube-api-access-cp8td\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787713 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-root\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787789 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-wtmp\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787961 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.787961 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.787847 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-sys\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.888842 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888759 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.888842 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888813 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-wtmp\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888863 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888887 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-sys\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:41.888913 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888985 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-sys\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:54:41.889006 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls podName:f1985411-2d30-4df7-9e4d-0208bf97dcf4 nodeName:}" failed. No retries permitted until 2026-05-28 20:54:42.388986264 +0000 UTC m=+170.004820961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls") pod "node-exporter-n2jbk" (UID: "f1985411-2d30-4df7-9e4d-0208bf97dcf4") : secret "node-exporter-tls" not found May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.888921 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-textfile\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889029 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-wtmp\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889053 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889056 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-metrics-client-ca\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889083 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889111 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cp8td\" (UniqueName: \"kubernetes.io/projected/f1985411-2d30-4df7-9e4d-0208bf97dcf4-kube-api-access-cp8td\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889163 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-root\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889241 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1985411-2d30-4df7-9e4d-0208bf97dcf4-root\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889396 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889248 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-textfile\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889617 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889601 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-metrics-client-ca\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.889687 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.889670 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.891134 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.891115 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:41.900036 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:41.900006 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp8td\" (UniqueName: \"kubernetes.io/projected/f1985411-2d30-4df7-9e4d-0208bf97dcf4-kube-api-access-cp8td\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:42.394326 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:42.394252 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:42.396417 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:42.396389 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1985411-2d30-4df7-9e4d-0208bf97dcf4-node-exporter-tls\") pod \"node-exporter-n2jbk\" (UID: \"f1985411-2d30-4df7-9e4d-0208bf97dcf4\") " pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:42.548890 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:42.548852 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n2jbk" May 28 20:54:42.557996 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:54:42.557964 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1985411_2d30_4df7_9e4d_0208bf97dcf4.slice/crio-3d58df8e8e6c184473597035f1f2b5cf91b18acba25c1fce6db031176ca263ff WatchSource:0}: Error finding container 3d58df8e8e6c184473597035f1f2b5cf91b18acba25c1fce6db031176ca263ff: Status 404 returned error can't find the container with id 3d58df8e8e6c184473597035f1f2b5cf91b18acba25c1fce6db031176ca263ff May 28 20:54:42.988700 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:42.988668 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:54:42.990736 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:42.990710 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gngfj\"" May 28 20:54:43.000036 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:43.000017 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vqhmn" May 28 20:54:43.131433 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:43.131402 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vqhmn"] May 28 20:54:43.135647 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:54:43.135616 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b62715f_a93e_4544_8a69_3d443630e2d2.slice/crio-ed1f69cd6df1b1eb826bbbc2334ae54d11c394f36087caf5e9ad239607821d6f WatchSource:0}: Error finding container ed1f69cd6df1b1eb826bbbc2334ae54d11c394f36087caf5e9ad239607821d6f: Status 404 returned error can't find the container with id ed1f69cd6df1b1eb826bbbc2334ae54d11c394f36087caf5e9ad239607821d6f May 28 20:54:43.482858 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:43.482810 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vqhmn" event={"ID":"9b62715f-a93e-4544-8a69-3d443630e2d2","Type":"ContainerStarted","Data":"ed1f69cd6df1b1eb826bbbc2334ae54d11c394f36087caf5e9ad239607821d6f"} May 28 20:54:43.483957 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:43.483915 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2jbk" event={"ID":"f1985411-2d30-4df7-9e4d-0208bf97dcf4","Type":"ContainerStarted","Data":"3d58df8e8e6c184473597035f1f2b5cf91b18acba25c1fce6db031176ca263ff"} May 28 20:54:43.987521 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:43.987487 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:54:44.487995 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:44.487958 2570 generic.go:358] "Generic (PLEG): container finished" podID="f1985411-2d30-4df7-9e4d-0208bf97dcf4" containerID="2073f064ccf45c0459acbc19511cf7a8a829a78f7e01f84c0a4eda568915df2e" exitCode=0 May 28 20:54:44.488402 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:44.488053 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2jbk" event={"ID":"f1985411-2d30-4df7-9e4d-0208bf97dcf4","Type":"ContainerDied","Data":"2073f064ccf45c0459acbc19511cf7a8a829a78f7e01f84c0a4eda568915df2e"} May 28 20:54:45.492525 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:45.492407 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vqhmn" event={"ID":"9b62715f-a93e-4544-8a69-3d443630e2d2","Type":"ContainerStarted","Data":"8fdf5f3b47fb0e5cbfd2a75241748db728826d2e21d7c5d9ebb99ab7d0f1a850"} May 28 20:54:45.494183 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:45.494160 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2jbk" event={"ID":"f1985411-2d30-4df7-9e4d-0208bf97dcf4","Type":"ContainerStarted","Data":"60518da3b34de55137dd78bce722cbbcf34416a2b4b6a9a71226035d9dd01d48"} May 28 20:54:45.494289 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:45.494188 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n2jbk" event={"ID":"f1985411-2d30-4df7-9e4d-0208bf97dcf4","Type":"ContainerStarted","Data":"66b247543d694c7762a151206b6c4eed6d9d7711f56140294a9d40c71c36dc3c"} May 28 20:54:45.510801 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:45.510748 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vqhmn" podStartSLOduration=136.447466185 podStartE2EDuration="2m18.510733023s" podCreationTimestamp="2026-05-28 20:52:27 +0000 UTC" firstStartedPulling="2026-05-28 20:54:43.138002774 +0000 UTC m=+170.753837477" lastFinishedPulling="2026-05-28 20:54:45.201269609 +0000 UTC m=+172.817104315" observedRunningTime="2026-05-28 20:54:45.51013404 +0000 UTC m=+173.125968770" watchObservedRunningTime="2026-05-28 20:54:45.510733023 +0000 UTC m=+173.126567744" May 28 20:54:45.526761 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:45.526714 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n2jbk" podStartSLOduration=3.531704527 podStartE2EDuration="4.526699197s" podCreationTimestamp="2026-05-28 20:54:41 +0000 UTC" firstStartedPulling="2026-05-28 20:54:42.560181961 +0000 UTC m=+170.176016658" lastFinishedPulling="2026-05-28 20:54:43.55517663 +0000 UTC m=+171.171011328" observedRunningTime="2026-05-28 20:54:45.525651111 +0000 UTC m=+173.141485854" watchObservedRunningTime="2026-05-28 20:54:45.526699197 +0000 UTC m=+173.142533921" May 28 20:54:48.345233 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:48.345199 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:54:48.472234 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:48.472201 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2nlt4" May 28 20:54:58.350793 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:54:58.350762 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:55:13.363618 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.363546 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" containerName="registry" containerID="cri-o://b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634" gracePeriod=30 May 28 20:55:13.396254 ip-10-0-136-77 kubenswrapper[2570]: E0528 20:55:13.396220 2570 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19b0e0c8_8889_49b2_babe_2762d09ad773.slice/crio-conmon-b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634.scope\": RecentStats: unable to find data in memory cache]" May 28 20:55:13.566561 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.566528 2570 generic.go:358] "Generic (PLEG): container finished" podID="19b0e0c8-8889-49b2-babe-2762d09ad773" containerID="b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634" exitCode=0 May 28 20:55:13.566759 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.566598 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" event={"ID":"19b0e0c8-8889-49b2-babe-2762d09ad773","Type":"ContainerDied","Data":"b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634"} May 28 20:55:13.604567 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.604544 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:55:13.631090 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631016 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631090 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631050 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631101 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631128 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631179 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631214 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631239 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631284 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631265 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4nlf\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf\") pod \"19b0e0c8-8889-49b2-babe-2762d09ad773\" (UID: \"19b0e0c8-8889-49b2-babe-2762d09ad773\") " May 28 20:55:13.631610 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631545 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:55:13.631675 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631646 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-certificates\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.631989 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.631951 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 28 20:55:13.634289 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.634258 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:55:13.634388 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.634310 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:55:13.634388 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.634318 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf" (OuterVolumeSpecName: "kube-api-access-r4nlf") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "kube-api-access-r4nlf". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:55:13.634523 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.634393 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 20:55:13.634782 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.634759 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 28 20:55:13.642760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.642733 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "19b0e0c8-8889-49b2-babe-2762d09ad773" (UID: "19b0e0c8-8889-49b2-babe-2762d09ad773"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 20:55:13.732299 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732261 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-bound-sa-token\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732299 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732295 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-installation-pull-secrets\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732502 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732310 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/19b0e0c8-8889-49b2-babe-2762d09ad773-trusted-ca\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732502 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732323 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4nlf\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-kube-api-access-r4nlf\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732502 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732335 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/19b0e0c8-8889-49b2-babe-2762d09ad773-registry-tls\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732502 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732347 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/19b0e0c8-8889-49b2-babe-2762d09ad773-ca-trust-extracted\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:13.732502 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:13.732358 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/19b0e0c8-8889-49b2-babe-2762d09ad773-image-registry-private-configuration\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 20:55:14.570042 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.570006 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" event={"ID":"19b0e0c8-8889-49b2-babe-2762d09ad773","Type":"ContainerDied","Data":"6e57426ec36b37f3d0c44261e1814326756467712bbd7010f878826318631362"} May 28 20:55:14.570042 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.570033 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-776cd9fc89-d9h22" May 28 20:55:14.570042 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.570046 2570 scope.go:117] "RemoveContainer" containerID="b6e90e70d19018d697bd5a7f9700a2be497423d9889c004e0403abcf98505634" May 28 20:55:14.591165 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.591133 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:55:14.596952 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.596926 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-776cd9fc89-d9h22"] May 28 20:55:14.990513 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:55:14.990428 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" path="/var/lib/kubelet/pods/19b0e0c8-8889-49b2-babe-2762d09ad773/volumes" May 28 20:56:04.799673 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:04.799632 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:56:04.801831 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:04.801812 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df288737-cf68-4b61-9f6b-4a87f5df1a98-metrics-certs\") pod \"network-metrics-daemon-mspmh\" (UID: \"df288737-cf68-4b61-9f6b-4a87f5df1a98\") " pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:56:04.990222 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:04.990190 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kbvdl\"" May 28 20:56:04.998656 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:04.998638 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mspmh" May 28 20:56:05.117155 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:05.117124 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mspmh"] May 28 20:56:05.120772 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:56:05.120745 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf288737_cf68_4b61_9f6b_4a87f5df1a98.slice/crio-f96b77540578ef0a1db313a9104516b82d33827e5885093687b4e9bd1d6bb417 WatchSource:0}: Error finding container f96b77540578ef0a1db313a9104516b82d33827e5885093687b4e9bd1d6bb417: Status 404 returned error can't find the container with id f96b77540578ef0a1db313a9104516b82d33827e5885093687b4e9bd1d6bb417 May 28 20:56:05.698373 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:05.698333 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mspmh" event={"ID":"df288737-cf68-4b61-9f6b-4a87f5df1a98","Type":"ContainerStarted","Data":"f96b77540578ef0a1db313a9104516b82d33827e5885093687b4e9bd1d6bb417"} May 28 20:56:06.703755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:06.703653 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mspmh" event={"ID":"df288737-cf68-4b61-9f6b-4a87f5df1a98","Type":"ContainerStarted","Data":"0cc0265a258d4e0dbb81d488aff57d39674ccd7c01c9ffe71ff9fb7bc7e829e8"} May 28 20:56:06.703755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:06.703693 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mspmh" event={"ID":"df288737-cf68-4b61-9f6b-4a87f5df1a98","Type":"ContainerStarted","Data":"2e51f9c11a0574ed6a37bea0583755df4f08b403b24d0b90c5d8c968f373d0be"} May 28 20:56:06.718712 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:06.718665 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mspmh" podStartSLOduration=252.570426764 podStartE2EDuration="4m13.718652552s" podCreationTimestamp="2026-05-28 20:51:53 +0000 UTC" firstStartedPulling="2026-05-28 20:56:05.122400776 +0000 UTC m=+252.738235473" lastFinishedPulling="2026-05-28 20:56:06.270626564 +0000 UTC m=+253.886461261" observedRunningTime="2026-05-28 20:56:06.717184537 +0000 UTC m=+254.333019269" watchObservedRunningTime="2026-05-28 20:56:06.718652552 +0000 UTC m=+254.334487272" May 28 20:56:52.906770 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:56:52.906745 2570 kubelet.go:1628] "Image garbage collection succeeded" May 28 20:59:13.858567 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.858488 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-xlzcx"] May 28 20:59:13.859001 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.858717 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" containerName="registry" May 28 20:59:13.859001 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.858728 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" containerName="registry" May 28 20:59:13.859001 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.858766 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="19b0e0c8-8889-49b2-babe-2762d09ad773" containerName="registry" May 28 20:59:13.861108 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.861088 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:13.863307 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.863288 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" May 28 20:59:13.863390 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.863301 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" May 28 20:59:13.863630 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.863613 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xsq2f\"" May 28 20:59:13.869670 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.869650 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-xlzcx"] May 28 20:59:13.995994 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.995965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:13.996160 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:13.996003 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlt96\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-kube-api-access-tlt96\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.096769 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.096736 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.096884 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.096777 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tlt96\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-kube-api-access-tlt96\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.104986 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.104952 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.105090 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.104953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlt96\" (UniqueName: \"kubernetes.io/projected/4df4dce2-931c-4f73-9533-499e8ee0f871-kube-api-access-tlt96\") pod \"cert-manager-cainjector-68b757865b-xlzcx\" (UID: \"4df4dce2-931c-4f73-9533-499e8ee0f871\") " pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.170369 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.170286 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" May 28 20:59:14.289251 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.289212 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-xlzcx"] May 28 20:59:14.292939 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:59:14.292912 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4df4dce2_931c_4f73_9533_499e8ee0f871.slice/crio-1654b570248d9e6dc631262c512fe00803d314732a7ade87f097cc45ee252405 WatchSource:0}: Error finding container 1654b570248d9e6dc631262c512fe00803d314732a7ade87f097cc45ee252405: Status 404 returned error can't find the container with id 1654b570248d9e6dc631262c512fe00803d314732a7ade87f097cc45ee252405 May 28 20:59:14.294717 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:14.294699 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 20:59:15.162343 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:15.162301 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" event={"ID":"4df4dce2-931c-4f73-9533-499e8ee0f871","Type":"ContainerStarted","Data":"1654b570248d9e6dc631262c512fe00803d314732a7ade87f097cc45ee252405"} May 28 20:59:17.169116 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:17.169075 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" event={"ID":"4df4dce2-931c-4f73-9533-499e8ee0f871","Type":"ContainerStarted","Data":"28292e7da8698d8c97dfb8355fad158c615a0cca87f8d64fac9f462ab8286ab0"} May 28 20:59:17.184040 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:17.183996 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-xlzcx" podStartSLOduration=2.057578749 podStartE2EDuration="4.183984134s" podCreationTimestamp="2026-05-28 20:59:13 +0000 UTC" firstStartedPulling="2026-05-28 20:59:14.294827796 +0000 UTC m=+441.910662494" lastFinishedPulling="2026-05-28 20:59:16.421233173 +0000 UTC m=+444.037067879" observedRunningTime="2026-05-28 20:59:17.183622432 +0000 UTC m=+444.799457151" watchObservedRunningTime="2026-05-28 20:59:17.183984134 +0000 UTC m=+444.799818854" May 28 20:59:21.097538 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.097505 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx"] May 28 20:59:21.100449 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.100433 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.102135 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.102113 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 28 20:59:21.102263 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.102153 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-97zvk\"" May 28 20:59:21.102630 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.102616 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 28 20:59:21.111084 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.111062 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx"] May 28 20:59:21.245639 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.245598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-tmp\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.245639 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.245640 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnzwg\" (UniqueName: \"kubernetes.io/projected/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-kube-api-access-cnzwg\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.346018 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.345988 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-tmp\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.346156 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.346026 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cnzwg\" (UniqueName: \"kubernetes.io/projected/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-kube-api-access-cnzwg\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.346379 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.346359 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-tmp\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.354060 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.354008 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnzwg\" (UniqueName: \"kubernetes.io/projected/a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb-kube-api-access-cnzwg\") pod \"openshift-lws-operator-bfc7f696d-gp8cx\" (UID: \"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.409593 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.409563 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" May 28 20:59:21.531128 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:21.531097 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx"] May 28 20:59:21.534655 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:59:21.534626 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda97ba9aa_4fa4_45e2_ad8a_293d5ed3f3fb.slice/crio-93bcfd4679c2728fcd6085839ca5d30bbc8d65319c5372738c356f3206d2fecc WatchSource:0}: Error finding container 93bcfd4679c2728fcd6085839ca5d30bbc8d65319c5372738c356f3206d2fecc: Status 404 returned error can't find the container with id 93bcfd4679c2728fcd6085839ca5d30bbc8d65319c5372738c356f3206d2fecc May 28 20:59:22.183157 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:22.183119 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" event={"ID":"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb","Type":"ContainerStarted","Data":"93bcfd4679c2728fcd6085839ca5d30bbc8d65319c5372738c356f3206d2fecc"} May 28 20:59:25.191366 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:25.191331 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" event={"ID":"a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb","Type":"ContainerStarted","Data":"7cd6c9bf437965f0801e6e8abd848bb2faedcda427d47641db47fe4fe93ddf87"} May 28 20:59:25.207107 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:25.207046 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-gp8cx" podStartSLOduration=1.409724031 podStartE2EDuration="4.207029992s" podCreationTimestamp="2026-05-28 20:59:21 +0000 UTC" firstStartedPulling="2026-05-28 20:59:21.536035497 +0000 UTC m=+449.151870195" lastFinishedPulling="2026-05-28 20:59:24.333341458 +0000 UTC m=+451.949176156" observedRunningTime="2026-05-28 20:59:25.205903167 +0000 UTC m=+452.821737886" watchObservedRunningTime="2026-05-28 20:59:25.207029992 +0000 UTC m=+452.822864713" May 28 20:59:44.174760 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.174728 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj"] May 28 20:59:44.177755 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.177736 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.185288 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.185266 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 28 20:59:44.185288 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.185282 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 28 20:59:44.185458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.185266 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 28 20:59:44.185458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.185339 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s2tr4\"" May 28 20:59:44.185458 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.185339 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 28 20:59:44.194169 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.194146 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj"] May 28 20:59:44.309963 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.309933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.310117 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.309987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndch\" (UniqueName: \"kubernetes.io/projected/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-kube-api-access-7ndch\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.310117 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.310073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.410768 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.410738 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.410895 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.410778 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndch\" (UniqueName: \"kubernetes.io/projected/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-kube-api-access-7ndch\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.410895 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.410805 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.413091 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.413063 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-webhook-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.413199 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.413095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.423398 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.423376 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndch\" (UniqueName: \"kubernetes.io/projected/d5c11e9a-45ab-4ccf-bc86-d72776d29d60-kube-api-access-7ndch\") pod \"opendatahub-operator-controller-manager-6dc97fc4b4-8pznj\" (UID: \"d5c11e9a-45ab-4ccf-bc86-d72776d29d60\") " pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.488407 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.488340 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:44.614040 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:44.613964 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj"] May 28 20:59:44.617784 ip-10-0-136-77 kubenswrapper[2570]: W0528 20:59:44.617757 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c11e9a_45ab_4ccf_bc86_d72776d29d60.slice/crio-be96042907afec493b58c5a9d3e1ba588e076fa77038713148a4b07abd18f9d0 WatchSource:0}: Error finding container be96042907afec493b58c5a9d3e1ba588e076fa77038713148a4b07abd18f9d0: Status 404 returned error can't find the container with id be96042907afec493b58c5a9d3e1ba588e076fa77038713148a4b07abd18f9d0 May 28 20:59:45.244182 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:45.244120 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" event={"ID":"d5c11e9a-45ab-4ccf-bc86-d72776d29d60","Type":"ContainerStarted","Data":"be96042907afec493b58c5a9d3e1ba588e076fa77038713148a4b07abd18f9d0"} May 28 20:59:48.254983 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:48.254947 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" event={"ID":"d5c11e9a-45ab-4ccf-bc86-d72776d29d60","Type":"ContainerStarted","Data":"06090f92386e21d695df3214c76c3c0f9f0500321e6faf0a659e8ce0738634db"} May 28 20:59:48.255375 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:48.255102 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 20:59:48.353310 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:48.353260 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" podStartSLOduration=1.54890937 podStartE2EDuration="4.35324673s" podCreationTimestamp="2026-05-28 20:59:44 +0000 UTC" firstStartedPulling="2026-05-28 20:59:44.619651151 +0000 UTC m=+472.235485849" lastFinishedPulling="2026-05-28 20:59:47.423988493 +0000 UTC m=+475.039823209" observedRunningTime="2026-05-28 20:59:48.352610278 +0000 UTC m=+475.968445023" watchObservedRunningTime="2026-05-28 20:59:48.35324673 +0000 UTC m=+475.969081450" May 28 20:59:59.261089 ip-10-0-136-77 kubenswrapper[2570]: I0528 20:59:59.261048 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6dc97fc4b4-8pznj" May 28 21:00:02.939313 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.939271 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6"] May 28 21:00:02.942620 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.942599 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:02.944366 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.944348 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 28 21:00:02.944813 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.944791 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 28 21:00:02.944813 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.944808 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-dtjbd\"" May 28 21:00:02.944936 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.944815 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 28 21:00:02.944936 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.944826 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 28 21:00:02.953330 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.953309 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6"] May 28 21:00:02.954489 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.954451 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:02.954579 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.954531 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sgh8\" (UniqueName: \"kubernetes.io/projected/6e662cd7-de9c-4e54-8fc2-9993a6397b18-kube-api-access-8sgh8\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:02.954579 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:02.954562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tmp\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.055288 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.055242 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tmp\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.055288 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.055299 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.055521 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.055359 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8sgh8\" (UniqueName: \"kubernetes.io/projected/6e662cd7-de9c-4e54-8fc2-9993a6397b18-kube-api-access-8sgh8\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.057603 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.057576 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tmp\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.057848 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.057829 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6e662cd7-de9c-4e54-8fc2-9993a6397b18-tls-certs\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.062724 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.062705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sgh8\" (UniqueName: \"kubernetes.io/projected/6e662cd7-de9c-4e54-8fc2-9993a6397b18-kube-api-access-8sgh8\") pod \"kube-auth-proxy-774cfddf4d-r5mn6\" (UID: \"6e662cd7-de9c-4e54-8fc2-9993a6397b18\") " pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.251386 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.251354 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" May 28 21:00:03.375419 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:03.375387 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6"] May 28 21:00:03.378381 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:00:03.378347 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e662cd7_de9c_4e54_8fc2_9993a6397b18.slice/crio-4eae432b1838f4e0ec0d96c55ebdc712d52228f187343c6bfd5c419e7cdb6412 WatchSource:0}: Error finding container 4eae432b1838f4e0ec0d96c55ebdc712d52228f187343c6bfd5c419e7cdb6412: Status 404 returned error can't find the container with id 4eae432b1838f4e0ec0d96c55ebdc712d52228f187343c6bfd5c419e7cdb6412 May 28 21:00:04.298241 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:04.298202 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" event={"ID":"6e662cd7-de9c-4e54-8fc2-9993a6397b18","Type":"ContainerStarted","Data":"4eae432b1838f4e0ec0d96c55ebdc712d52228f187343c6bfd5c419e7cdb6412"} May 28 21:00:05.465804 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.465769 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7j9td"] May 28 21:00:05.467681 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.467665 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:05.469631 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.469463 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4xm96\"" May 28 21:00:05.469631 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.469523 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 28 21:00:05.478484 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.478445 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7j9td"] May 28 21:00:05.574102 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.574075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:05.574256 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.574140 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjwdr\" (UniqueName: \"kubernetes.io/projected/499ea2a9-9c07-4025-a40c-8ac7686f3101-kube-api-access-rjwdr\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:05.675434 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.675396 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjwdr\" (UniqueName: \"kubernetes.io/projected/499ea2a9-9c07-4025-a40c-8ac7686f3101-kube-api-access-rjwdr\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:05.675434 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.675437 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:05.675667 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:05.675565 2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 28 21:00:05.675667 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:05.675629 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert podName:499ea2a9-9c07-4025-a40c-8ac7686f3101 nodeName:}" failed. No retries permitted until 2026-05-28 21:00:06.17560777 +0000 UTC m=+493.791442475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert") pod "odh-model-controller-858dbf95b8-7j9td" (UID: "499ea2a9-9c07-4025-a40c-8ac7686f3101") : secret "odh-model-controller-webhook-cert" not found May 28 21:00:05.683800 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:05.683769 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjwdr\" (UniqueName: \"kubernetes.io/projected/499ea2a9-9c07-4025-a40c-8ac7686f3101-kube-api-access-rjwdr\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:06.179140 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:06.179102 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:06.181522 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:06.181495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/499ea2a9-9c07-4025-a40c-8ac7686f3101-cert\") pod \"odh-model-controller-858dbf95b8-7j9td\" (UID: \"499ea2a9-9c07-4025-a40c-8ac7686f3101\") " pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:06.378708 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:06.378666 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:06.960464 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:06.960434 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-7j9td"] May 28 21:00:06.964589 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:00:06.964563 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod499ea2a9_9c07_4025_a40c_8ac7686f3101.slice/crio-a28aed88e57c4fc301bec8b7930d4f4d6616dfc2753a3c02c2e4643402fe1479 WatchSource:0}: Error finding container a28aed88e57c4fc301bec8b7930d4f4d6616dfc2753a3c02c2e4643402fe1479: Status 404 returned error can't find the container with id a28aed88e57c4fc301bec8b7930d4f4d6616dfc2753a3c02c2e4643402fe1479 May 28 21:00:07.308067 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:07.308033 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" event={"ID":"499ea2a9-9c07-4025-a40c-8ac7686f3101","Type":"ContainerStarted","Data":"a28aed88e57c4fc301bec8b7930d4f4d6616dfc2753a3c02c2e4643402fe1479"} May 28 21:00:07.309302 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:07.309274 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" event={"ID":"6e662cd7-de9c-4e54-8fc2-9993a6397b18","Type":"ContainerStarted","Data":"369b15ccb98b3485396d8311ed536334b2cdee4c70cd9b4d4f06c6e96d6134b3"} May 28 21:00:07.329872 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:07.329825 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-774cfddf4d-r5mn6" podStartSLOduration=1.8319053140000001 podStartE2EDuration="5.329810487s" podCreationTimestamp="2026-05-28 21:00:02 +0000 UTC" firstStartedPulling="2026-05-28 21:00:03.380057736 +0000 UTC m=+490.995892433" lastFinishedPulling="2026-05-28 21:00:06.877962906 +0000 UTC m=+494.493797606" observedRunningTime="2026-05-28 21:00:07.327869119 +0000 UTC m=+494.943703838" watchObservedRunningTime="2026-05-28 21:00:07.329810487 +0000 UTC m=+494.945645206" May 28 21:00:10.319453 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:10.319413 2570 generic.go:358] "Generic (PLEG): container finished" podID="499ea2a9-9c07-4025-a40c-8ac7686f3101" containerID="e4bda620d01026ff5ff1ba848926d1ced4bc7615e0e57193b8af3f44f0f20253" exitCode=1 May 28 21:00:10.319946 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:10.319493 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" event={"ID":"499ea2a9-9c07-4025-a40c-8ac7686f3101","Type":"ContainerDied","Data":"e4bda620d01026ff5ff1ba848926d1ced4bc7615e0e57193b8af3f44f0f20253"} May 28 21:00:10.319946 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:10.319770 2570 scope.go:117] "RemoveContainer" containerID="e4bda620d01026ff5ff1ba848926d1ced4bc7615e0e57193b8af3f44f0f20253" May 28 21:00:11.277130 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.277096 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-dtlmn"] May 28 21:00:11.279194 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.279176 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.280936 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.280917 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 28 21:00:11.281024 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.281007 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wxf2d\"" May 28 21:00:11.288499 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.288457 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-dtlmn"] May 28 21:00:11.321270 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.321243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66f4t\" (UniqueName: \"kubernetes.io/projected/dacdada3-fb15-43b7-9a78-fc447ad75911-kube-api-access-66f4t\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.321728 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.321322 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.324335 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.324312 2570 generic.go:358] "Generic (PLEG): container finished" podID="499ea2a9-9c07-4025-a40c-8ac7686f3101" containerID="77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e" exitCode=1 May 28 21:00:11.324442 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.324372 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" event={"ID":"499ea2a9-9c07-4025-a40c-8ac7686f3101","Type":"ContainerDied","Data":"77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e"} May 28 21:00:11.324442 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.324402 2570 scope.go:117] "RemoveContainer" containerID="e4bda620d01026ff5ff1ba848926d1ced4bc7615e0e57193b8af3f44f0f20253" May 28 21:00:11.324674 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.324654 2570 scope.go:117] "RemoveContainer" containerID="77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e" May 28 21:00:11.324864 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:11.324847 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-7j9td_opendatahub(499ea2a9-9c07-4025-a40c-8ac7686f3101)\"" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" podUID="499ea2a9-9c07-4025-a40c-8ac7686f3101" May 28 21:00:11.421687 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.421649 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.421866 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:11.421798 2570 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 28 21:00:11.421866 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.421813 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-66f4t\" (UniqueName: \"kubernetes.io/projected/dacdada3-fb15-43b7-9a78-fc447ad75911-kube-api-access-66f4t\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.421963 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:11.421879 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert podName:dacdada3-fb15-43b7-9a78-fc447ad75911 nodeName:}" failed. No retries permitted until 2026-05-28 21:00:11.921860032 +0000 UTC m=+499.537694749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert") pod "kserve-controller-manager-856948b99f-dtlmn" (UID: "dacdada3-fb15-43b7-9a78-fc447ad75911") : secret "kserve-webhook-server-cert" not found May 28 21:00:11.438108 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.438081 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66f4t\" (UniqueName: \"kubernetes.io/projected/dacdada3-fb15-43b7-9a78-fc447ad75911-kube-api-access-66f4t\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.924855 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:11.924816 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:11.925029 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:11.924938 2570 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 28 21:00:11.925029 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:11.925008 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert podName:dacdada3-fb15-43b7-9a78-fc447ad75911 nodeName:}" failed. No retries permitted until 2026-05-28 21:00:12.924989665 +0000 UTC m=+500.540824363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert") pod "kserve-controller-manager-856948b99f-dtlmn" (UID: "dacdada3-fb15-43b7-9a78-fc447ad75911") : secret "kserve-webhook-server-cert" not found May 28 21:00:12.327962 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:12.327928 2570 scope.go:117] "RemoveContainer" containerID="77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e" May 28 21:00:12.328377 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:12.328131 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-7j9td_opendatahub(499ea2a9-9c07-4025-a40c-8ac7686f3101)\"" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" podUID="499ea2a9-9c07-4025-a40c-8ac7686f3101" May 28 21:00:12.932097 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:12.932058 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:12.934380 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:12.934357 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dacdada3-fb15-43b7-9a78-fc447ad75911-cert\") pod \"kserve-controller-manager-856948b99f-dtlmn\" (UID: \"dacdada3-fb15-43b7-9a78-fc447ad75911\") " pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:13.090177 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:13.090143 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:13.212587 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:13.212518 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-dtlmn"] May 28 21:00:13.215570 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:00:13.215538 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacdada3_fb15_43b7_9a78_fc447ad75911.slice/crio-98d16c4f91b71efbc7ce8cea0ce8667887f19dd1247fadd7d4bc99b38f285d10 WatchSource:0}: Error finding container 98d16c4f91b71efbc7ce8cea0ce8667887f19dd1247fadd7d4bc99b38f285d10: Status 404 returned error can't find the container with id 98d16c4f91b71efbc7ce8cea0ce8667887f19dd1247fadd7d4bc99b38f285d10 May 28 21:00:13.332127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:13.332093 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" event={"ID":"dacdada3-fb15-43b7-9a78-fc447ad75911","Type":"ContainerStarted","Data":"98d16c4f91b71efbc7ce8cea0ce8667887f19dd1247fadd7d4bc99b38f285d10"} May 28 21:00:16.379296 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:16.379256 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:16.379772 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:16.379702 2570 scope.go:117] "RemoveContainer" containerID="77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e" May 28 21:00:16.379899 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:00:16.379883 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-858dbf95b8-7j9td_opendatahub(499ea2a9-9c07-4025-a40c-8ac7686f3101)\"" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" podUID="499ea2a9-9c07-4025-a40c-8ac7686f3101" May 28 21:00:18.348767 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:18.348732 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" event={"ID":"dacdada3-fb15-43b7-9a78-fc447ad75911","Type":"ContainerStarted","Data":"720b98fc25dda112d0b389f717809f6bd2372b8739dd0e7c62bd28850cef1614"} May 28 21:00:18.349157 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:18.348834 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:18.366259 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:18.366209 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" podStartSLOduration=2.868933393 podStartE2EDuration="7.366194315s" podCreationTimestamp="2026-05-28 21:00:11 +0000 UTC" firstStartedPulling="2026-05-28 21:00:13.216887963 +0000 UTC m=+500.832722661" lastFinishedPulling="2026-05-28 21:00:17.714148882 +0000 UTC m=+505.329983583" observedRunningTime="2026-05-28 21:00:18.364811096 +0000 UTC m=+505.980645818" watchObservedRunningTime="2026-05-28 21:00:18.366194315 +0000 UTC m=+505.982029047" May 28 21:00:26.379640 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:26.379604 2570 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:26.380210 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:26.379970 2570 scope.go:117] "RemoveContainer" containerID="77ddfb165ee37fe819d55d174ff12e2ed040e33476ed4e6c16c86add1dc1880e" May 28 21:00:27.375620 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.375574 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" event={"ID":"499ea2a9-9c07-4025-a40c-8ac7686f3101","Type":"ContainerStarted","Data":"d3b42b6c769a49d81c6da75b8559d81e8a0b68cc24d32c37706f4ca6bef00bfd"} May 28 21:00:27.375806 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.375788 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:27.392324 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.392278 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" podStartSLOduration=2.7169156770000003 podStartE2EDuration="22.392265258s" podCreationTimestamp="2026-05-28 21:00:05 +0000 UTC" firstStartedPulling="2026-05-28 21:00:06.965930061 +0000 UTC m=+494.581764759" lastFinishedPulling="2026-05-28 21:00:26.641279643 +0000 UTC m=+514.257114340" observedRunningTime="2026-05-28 21:00:27.390833703 +0000 UTC m=+515.006668445" watchObservedRunningTime="2026-05-28 21:00:27.392265258 +0000 UTC m=+515.008099977" May 28 21:00:27.516383 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.516348 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-zghtx"] May 28 21:00:27.519858 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.519840 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.521670 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.521648 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 28 21:00:27.521780 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.521701 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 28 21:00:27.521780 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.521717 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-5h7n7\"" May 28 21:00:27.534719 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.534697 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-zghtx"] May 28 21:00:27.644577 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.644492 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/751a1f7b-e33f-4401-9dde-61feae26f0d2-operator-config\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.644577 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.644532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbhxh\" (UniqueName: \"kubernetes.io/projected/751a1f7b-e33f-4401-9dde-61feae26f0d2-kube-api-access-zbhxh\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.745662 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.745627 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/751a1f7b-e33f-4401-9dde-61feae26f0d2-operator-config\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.745662 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.745663 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zbhxh\" (UniqueName: \"kubernetes.io/projected/751a1f7b-e33f-4401-9dde-61feae26f0d2-kube-api-access-zbhxh\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.748110 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.748083 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/751a1f7b-e33f-4401-9dde-61feae26f0d2-operator-config\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.753270 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.753251 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbhxh\" (UniqueName: \"kubernetes.io/projected/751a1f7b-e33f-4401-9dde-61feae26f0d2-kube-api-access-zbhxh\") pod \"servicemesh-operator3-55f49c5f94-zghtx\" (UID: \"751a1f7b-e33f-4401-9dde-61feae26f0d2\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.829292 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.829265 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:27.968521 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:27.968486 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-zghtx"] May 28 21:00:27.972007 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:00:27.971980 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod751a1f7b_e33f_4401_9dde_61feae26f0d2.slice/crio-c42a78a7c2ca220ead4df5015f0457d802a6aabde6826f170f48103949cf8094 WatchSource:0}: Error finding container c42a78a7c2ca220ead4df5015f0457d802a6aabde6826f170f48103949cf8094: Status 404 returned error can't find the container with id c42a78a7c2ca220ead4df5015f0457d802a6aabde6826f170f48103949cf8094 May 28 21:00:28.379631 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:28.379594 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" event={"ID":"751a1f7b-e33f-4401-9dde-61feae26f0d2","Type":"ContainerStarted","Data":"c42a78a7c2ca220ead4df5015f0457d802a6aabde6826f170f48103949cf8094"} May 28 21:00:31.392184 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:31.392142 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" event={"ID":"751a1f7b-e33f-4401-9dde-61feae26f0d2","Type":"ContainerStarted","Data":"0e3f24055f40d11d5f21f7572f2d02e4df299a723067da5f3e58e7cb0bf60a97"} May 28 21:00:31.392679 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:31.392368 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:31.411704 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:31.411654 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" podStartSLOduration=1.8276431020000001 podStartE2EDuration="4.411638067s" podCreationTimestamp="2026-05-28 21:00:27 +0000 UTC" firstStartedPulling="2026-05-28 21:00:27.974636153 +0000 UTC m=+515.590470852" lastFinishedPulling="2026-05-28 21:00:30.558631104 +0000 UTC m=+518.174465817" observedRunningTime="2026-05-28 21:00:31.408965123 +0000 UTC m=+519.024799853" watchObservedRunningTime="2026-05-28 21:00:31.411638067 +0000 UTC m=+519.027472788" May 28 21:00:38.382556 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:38.382522 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-7j9td" May 28 21:00:42.396856 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:42.396771 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-zghtx" May 28 21:00:47.858524 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.858437 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6"] May 28 21:00:47.861881 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.861863 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:47.863689 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.863669 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 28 21:00:47.863808 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.863686 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 28 21:00:47.863808 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.863710 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hb6tc\"" May 28 21:00:47.863808 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.863728 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 28 21:00:47.863808 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.863693 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 28 21:00:47.872192 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:47.872164 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6"] May 28 21:00:48.006393 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006357 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-cacerts\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006393 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006399 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006637 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006637 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006447 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006637 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ef0a7c-bf5d-4171-b347-68003c93c5f1-local-certs\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006637 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-token\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.006637 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.006563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlw7\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-kube-api-access-pvlw7\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107437 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107396 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlw7\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-kube-api-access-pvlw7\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107437 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107443 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-cacerts\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107702 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107465 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107702 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107519 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107702 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107537 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107702 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107561 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ef0a7c-bf5d-4171-b347-68003c93c5f1-local-certs\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.107702 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.107584 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-token\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.108368 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.108343 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.110028 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.109961 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.110282 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.110261 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/36ef0a7c-bf5d-4171-b347-68003c93c5f1-local-certs\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.110349 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.110331 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-cacerts\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.110391 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.110379 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.114943 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.114918 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlw7\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-kube-api-access-pvlw7\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.115110 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.115088 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36ef0a7c-bf5d-4171-b347-68003c93c5f1-istio-token\") pod \"istiod-openshift-gateway-798958bb55-d5bh6\" (UID: \"36ef0a7c-bf5d-4171-b347-68003c93c5f1\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.171105 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.171072 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:48.300786 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.300757 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6"] May 28 21:00:48.303607 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:00:48.303579 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36ef0a7c_bf5d_4171_b347_68003c93c5f1.slice/crio-65b053980f9b50f0da0db519457256cdd97a9c5901a1efbb8ea583c14144bac7 WatchSource:0}: Error finding container 65b053980f9b50f0da0db519457256cdd97a9c5901a1efbb8ea583c14144bac7: Status 404 returned error can't find the container with id 65b053980f9b50f0da0db519457256cdd97a9c5901a1efbb8ea583c14144bac7 May 28 21:00:48.442966 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:48.442873 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" event={"ID":"36ef0a7c-bf5d-4171-b347-68003c93c5f1","Type":"ContainerStarted","Data":"65b053980f9b50f0da0db519457256cdd97a9c5901a1efbb8ea583c14144bac7"} May 28 21:00:49.356357 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:49.356326 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-dtlmn" May 28 21:00:51.312242 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:51.312175 2570 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 28 21:00:51.312543 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:51.312288 2570 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} May 28 21:00:51.455401 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:51.455367 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" event={"ID":"36ef0a7c-bf5d-4171-b347-68003c93c5f1","Type":"ContainerStarted","Data":"4070a4ef1045d5a8d080ea62d753e447577153903b1069c2bf1e4fdca5b3dd3b"} May 28 21:00:51.455594 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:51.455418 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:52.460575 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:52.460542 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-d5bh6 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= May 28 21:00:52.460920 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:52.460606 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" podUID="36ef0a7c-bf5d-4171-b347-68003c93c5f1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 28 21:00:55.460182 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:55.460156 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" May 28 21:00:55.479349 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:00:55.479286 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-d5bh6" podStartSLOduration=5.4729305 podStartE2EDuration="8.479268399s" podCreationTimestamp="2026-05-28 21:00:47 +0000 UTC" firstStartedPulling="2026-05-28 21:00:48.305586209 +0000 UTC m=+535.921420910" lastFinishedPulling="2026-05-28 21:00:51.311924109 +0000 UTC m=+538.927758809" observedRunningTime="2026-05-28 21:00:51.473389997 +0000 UTC m=+539.089224717" watchObservedRunningTime="2026-05-28 21:00:55.479268399 +0000 UTC m=+543.095103120" May 28 21:01:41.977214 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.977173 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wv8zl"] May 28 21:01:41.984987 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.984961 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:41.987220 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.987197 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bxq7l\"" May 28 21:01:41.987916 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.987903 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 28 21:01:41.987989 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.987927 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 28 21:01:41.993293 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:41.993264 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wv8zl"] May 28 21:01:42.126187 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.126146 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvgxd\" (UniqueName: \"kubernetes.io/projected/09eb92c6-17f6-4064-9996-fd228ce35b3e-kube-api-access-xvgxd\") pod \"authorino-operator-657f44b778-wv8zl\" (UID: \"09eb92c6-17f6-4064-9996-fd228ce35b3e\") " pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:42.227594 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.227503 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xvgxd\" (UniqueName: \"kubernetes.io/projected/09eb92c6-17f6-4064-9996-fd228ce35b3e-kube-api-access-xvgxd\") pod \"authorino-operator-657f44b778-wv8zl\" (UID: \"09eb92c6-17f6-4064-9996-fd228ce35b3e\") " pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:42.244003 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.243971 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvgxd\" (UniqueName: \"kubernetes.io/projected/09eb92c6-17f6-4064-9996-fd228ce35b3e-kube-api-access-xvgxd\") pod \"authorino-operator-657f44b778-wv8zl\" (UID: \"09eb92c6-17f6-4064-9996-fd228ce35b3e\") " pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:42.295570 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.295539 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:42.430749 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.430699 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-wv8zl"] May 28 21:01:42.435884 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:01:42.435853 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09eb92c6_17f6_4064_9996_fd228ce35b3e.slice/crio-08ab35860895fcde7359137d35ae6271e9d6d77e8ffec7e73637cb05b9223839 WatchSource:0}: Error finding container 08ab35860895fcde7359137d35ae6271e9d6d77e8ffec7e73637cb05b9223839: Status 404 returned error can't find the container with id 08ab35860895fcde7359137d35ae6271e9d6d77e8ffec7e73637cb05b9223839 May 28 21:01:42.616024 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:42.615989 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" event={"ID":"09eb92c6-17f6-4064-9996-fd228ce35b3e","Type":"ContainerStarted","Data":"08ab35860895fcde7359137d35ae6271e9d6d77e8ffec7e73637cb05b9223839"} May 28 21:01:44.626230 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:44.626191 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" event={"ID":"09eb92c6-17f6-4064-9996-fd228ce35b3e","Type":"ContainerStarted","Data":"39f563104b1a8f0dce4b38f3aa92de4c5bd9f0b14b703b8487cbeb297ffc288b"} May 28 21:01:44.626634 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:44.626314 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:44.640684 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:44.640611 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" podStartSLOduration=1.612198102 podStartE2EDuration="3.640594524s" podCreationTimestamp="2026-05-28 21:01:41 +0000 UTC" firstStartedPulling="2026-05-28 21:01:42.437839723 +0000 UTC m=+590.053674426" lastFinishedPulling="2026-05-28 21:01:44.466236146 +0000 UTC m=+592.082070848" observedRunningTime="2026-05-28 21:01:44.639742925 +0000 UTC m=+592.255577657" watchObservedRunningTime="2026-05-28 21:01:44.640594524 +0000 UTC m=+592.256429246" May 28 21:01:55.631082 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:55.631052 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-wv8zl" May 28 21:01:58.215087 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.215058 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:01:58.218311 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.218294 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.220243 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.220218 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t4hzt\"" May 28 21:01:58.232175 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.232151 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:01:58.365154 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.365122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.365320 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.365223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw5nx\" (UniqueName: \"kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.466196 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.466116 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hw5nx\" (UniqueName: \"kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.466196 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.466160 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.466513 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.466496 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.474434 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.474412 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw5nx\" (UniqueName: \"kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx\") pod \"kuadrant-operator-controller-manager-55c7f4c975-745mp\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.527946 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.527923 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:01:58.663437 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.663406 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:01:58.666390 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:01:58.666359 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdba32b_b267_4b88_af74_51efb4a4bfe1.slice/crio-ee9eb0078ee84dfd96fa3abde3a5b3634e77e17dc65f40e0d08eaefe5d6c8257 WatchSource:0}: Error finding container ee9eb0078ee84dfd96fa3abde3a5b3634e77e17dc65f40e0d08eaefe5d6c8257: Status 404 returned error can't find the container with id ee9eb0078ee84dfd96fa3abde3a5b3634e77e17dc65f40e0d08eaefe5d6c8257 May 28 21:01:58.673854 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:01:58.673828 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" event={"ID":"0bdba32b-b267-4b88-af74-51efb4a4bfe1","Type":"ContainerStarted","Data":"ee9eb0078ee84dfd96fa3abde3a5b3634e77e17dc65f40e0d08eaefe5d6c8257"} May 28 21:02:03.693137 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:02:03.693104 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" event={"ID":"0bdba32b-b267-4b88-af74-51efb4a4bfe1","Type":"ContainerStarted","Data":"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78"} May 28 21:02:03.693558 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:02:03.693222 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:02:03.710840 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:02:03.710790 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" podStartSLOduration=0.786106742 podStartE2EDuration="5.710773621s" podCreationTimestamp="2026-05-28 21:01:58 +0000 UTC" firstStartedPulling="2026-05-28 21:01:58.670733389 +0000 UTC m=+606.286568088" lastFinishedPulling="2026-05-28 21:02:03.595400269 +0000 UTC m=+611.211234967" observedRunningTime="2026-05-28 21:02:03.709442223 +0000 UTC m=+611.325276942" watchObservedRunningTime="2026-05-28 21:02:03.710773621 +0000 UTC m=+611.326608343" May 28 21:02:14.698910 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:02:14.698831 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:04:01.794084 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.794004 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:01.796337 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.796305 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:01.798140 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.798117 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 28 21:04:01.798258 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.798182 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2wqh5\"" May 28 21:04:01.806030 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.806004 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:01.821519 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.821490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlhbs\" (UniqueName: \"kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs\") pod \"maas-controller-555455f4c7-vvvww\" (UID: \"49c296c7-966b-48af-8f8f-7ca592f61bff\") " pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:01.922188 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.922155 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rlhbs\" (UniqueName: \"kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs\") pod \"maas-controller-555455f4c7-vvvww\" (UID: \"49c296c7-966b-48af-8f8f-7ca592f61bff\") " pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:01.932683 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.932655 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlhbs\" (UniqueName: \"kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs\") pod \"maas-controller-555455f4c7-vvvww\" (UID: \"49c296c7-966b-48af-8f8f-7ca592f61bff\") " pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:01.941747 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.941722 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:04:01.943991 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.943976 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:01.958063 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:01.958038 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:04:02.022754 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.022720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqvpc\" (UniqueName: \"kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc\") pod \"maas-controller-764c88474f-rg8gk\" (UID: \"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f\") " pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:02.107825 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.107740 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:02.123763 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.123738 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rqvpc\" (UniqueName: \"kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc\") pod \"maas-controller-764c88474f-rg8gk\" (UID: \"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f\") " pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:02.131074 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.131052 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqvpc\" (UniqueName: \"kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc\") pod \"maas-controller-764c88474f-rg8gk\" (UID: \"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f\") " pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:02.235394 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.235372 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:02.238013 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:04:02.237977 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c296c7_966b_48af_8f8f_7ca592f61bff.slice/crio-369e119fefc25c5f689edb01d993609793de73ecf08b9921904e3038be3f3881 WatchSource:0}: Error finding container 369e119fefc25c5f689edb01d993609793de73ecf08b9921904e3038be3f3881: Status 404 returned error can't find the container with id 369e119fefc25c5f689edb01d993609793de73ecf08b9921904e3038be3f3881 May 28 21:04:02.253998 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.253970 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:02.372713 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:02.372635 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:04:02.375748 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:04:02.375724 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4c24d2_efe9_4ad5_89a0_4ebd8b9bd00f.slice/crio-dee5a3f076812c35027e90597f26df9e52604d720989f40587aabf4a5ef01839 WatchSource:0}: Error finding container dee5a3f076812c35027e90597f26df9e52604d720989f40587aabf4a5ef01839: Status 404 returned error can't find the container with id dee5a3f076812c35027e90597f26df9e52604d720989f40587aabf4a5ef01839 May 28 21:04:03.072779 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:03.072623 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-555455f4c7-vvvww" event={"ID":"49c296c7-966b-48af-8f8f-7ca592f61bff","Type":"ContainerStarted","Data":"369e119fefc25c5f689edb01d993609793de73ecf08b9921904e3038be3f3881"} May 28 21:04:03.075557 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:03.075372 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-rg8gk" event={"ID":"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f","Type":"ContainerStarted","Data":"dee5a3f076812c35027e90597f26df9e52604d720989f40587aabf4a5ef01839"} May 28 21:04:05.084348 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.084309 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-555455f4c7-vvvww" event={"ID":"49c296c7-966b-48af-8f8f-7ca592f61bff","Type":"ContainerStarted","Data":"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7"} May 28 21:04:05.084817 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.084440 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:05.085723 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.085699 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-rg8gk" event={"ID":"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f","Type":"ContainerStarted","Data":"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8"} May 28 21:04:05.085838 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.085796 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:05.104572 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.104530 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-555455f4c7-vvvww" podStartSLOduration=1.474656622 podStartE2EDuration="4.104516777s" podCreationTimestamp="2026-05-28 21:04:01 +0000 UTC" firstStartedPulling="2026-05-28 21:04:02.239298577 +0000 UTC m=+729.855133275" lastFinishedPulling="2026-05-28 21:04:04.869158722 +0000 UTC m=+732.484993430" observedRunningTime="2026-05-28 21:04:05.102216396 +0000 UTC m=+732.718051117" watchObservedRunningTime="2026-05-28 21:04:05.104516777 +0000 UTC m=+732.720351497" May 28 21:04:05.120214 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:05.120113 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-764c88474f-rg8gk" podStartSLOduration=1.636444765 podStartE2EDuration="4.120098042s" podCreationTimestamp="2026-05-28 21:04:01 +0000 UTC" firstStartedPulling="2026-05-28 21:04:02.376913937 +0000 UTC m=+729.992748636" lastFinishedPulling="2026-05-28 21:04:04.860567199 +0000 UTC m=+732.476401913" observedRunningTime="2026-05-28 21:04:05.118192384 +0000 UTC m=+732.734027105" watchObservedRunningTime="2026-05-28 21:04:05.120098042 +0000 UTC m=+732.735932762" May 28 21:04:16.093990 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.093949 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:04:16.094599 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.094016 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:16.146463 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.146435 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:16.146659 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.146639 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-555455f4c7-vvvww" podUID="49c296c7-966b-48af-8f8f-7ca592f61bff" containerName="manager" containerID="cri-o://395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7" gracePeriod=30 May 28 21:04:16.390798 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.390776 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:16.440886 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.440857 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlhbs\" (UniqueName: \"kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs\") pod \"49c296c7-966b-48af-8f8f-7ca592f61bff\" (UID: \"49c296c7-966b-48af-8f8f-7ca592f61bff\") " May 28 21:04:16.442993 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.442963 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs" (OuterVolumeSpecName: "kube-api-access-rlhbs") pod "49c296c7-966b-48af-8f8f-7ca592f61bff" (UID: "49c296c7-966b-48af-8f8f-7ca592f61bff"). InnerVolumeSpecName "kube-api-access-rlhbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:04:16.542295 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:16.542258 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlhbs\" (UniqueName: \"kubernetes.io/projected/49c296c7-966b-48af-8f8f-7ca592f61bff-kube-api-access-rlhbs\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 21:04:17.122329 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.122293 2570 generic.go:358] "Generic (PLEG): container finished" podID="49c296c7-966b-48af-8f8f-7ca592f61bff" containerID="395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7" exitCode=0 May 28 21:04:17.122777 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.122354 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-555455f4c7-vvvww" May 28 21:04:17.122777 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.122378 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-555455f4c7-vvvww" event={"ID":"49c296c7-966b-48af-8f8f-7ca592f61bff","Type":"ContainerDied","Data":"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7"} May 28 21:04:17.122777 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.122418 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-555455f4c7-vvvww" event={"ID":"49c296c7-966b-48af-8f8f-7ca592f61bff","Type":"ContainerDied","Data":"369e119fefc25c5f689edb01d993609793de73ecf08b9921904e3038be3f3881"} May 28 21:04:17.122777 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.122433 2570 scope.go:117] "RemoveContainer" containerID="395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7" May 28 21:04:17.129918 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.129891 2570 scope.go:117] "RemoveContainer" containerID="395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7" May 28 21:04:17.130174 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:04:17.130150 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7\": container with ID starting with 395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7 not found: ID does not exist" containerID="395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7" May 28 21:04:17.130249 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.130181 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7"} err="failed to get container status \"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7\": rpc error: code = NotFound desc = could not find container \"395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7\": container with ID starting with 395e9a915c54d4757ed76fa47de3e65b4a53824f092452309e5d9a172a1354f7 not found: ID does not exist" May 28 21:04:17.138779 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.138760 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:17.142769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:17.142746 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-555455f4c7-vvvww"] May 28 21:04:18.991555 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:18.991524 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c296c7-966b-48af-8f8f-7ca592f61bff" path="/var/lib/kubelet/pods/49c296c7-966b-48af-8f8f-7ca592f61bff/volumes" May 28 21:04:33.236047 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.236010 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-fc9cf5979-tx2b7"] May 28 21:04:33.236497 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.236334 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c296c7-966b-48af-8f8f-7ca592f61bff" containerName="manager" May 28 21:04:33.236497 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.236347 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c296c7-966b-48af-8f8f-7ca592f61bff" containerName="manager" May 28 21:04:33.236497 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.236394 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="49c296c7-966b-48af-8f8f-7ca592f61bff" containerName="manager" May 28 21:04:33.239708 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.239690 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.241212 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.241190 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 28 21:04:33.249205 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.249184 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-fc9cf5979-tx2b7"] May 28 21:04:33.378563 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.378523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkgc8\" (UniqueName: \"kubernetes.io/projected/7809dda2-6bba-4a4f-939c-aecf65f7f892-kube-api-access-rkgc8\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.378563 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.378570 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7809dda2-6bba-4a4f-939c-aecf65f7f892-maas-api-tls\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.479036 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.478993 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rkgc8\" (UniqueName: \"kubernetes.io/projected/7809dda2-6bba-4a4f-939c-aecf65f7f892-kube-api-access-rkgc8\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.479036 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.479041 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7809dda2-6bba-4a4f-939c-aecf65f7f892-maas-api-tls\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.481530 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.481506 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7809dda2-6bba-4a4f-939c-aecf65f7f892-maas-api-tls\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.486135 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.486078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkgc8\" (UniqueName: \"kubernetes.io/projected/7809dda2-6bba-4a4f-939c-aecf65f7f892-kube-api-access-rkgc8\") pod \"maas-api-fc9cf5979-tx2b7\" (UID: \"7809dda2-6bba-4a4f-939c-aecf65f7f892\") " pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.549585 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.549553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:33.679734 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.679688 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-r7dwz"] May 28 21:04:33.682621 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.682602 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:33.684623 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.684598 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 28 21:04:33.684716 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.684674 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-grzsm\"" May 28 21:04:33.688108 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.688076 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-r7dwz"] May 28 21:04:33.692886 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.692852 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-fc9cf5979-tx2b7"] May 28 21:04:33.696340 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:04:33.696316 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7809dda2_6bba_4a4f_939c_aecf65f7f892.slice/crio-884ebfbcb654723c19c7802d4c7986cab29f810566449fbdfdd94b4ae5969cef WatchSource:0}: Error finding container 884ebfbcb654723c19c7802d4c7986cab29f810566449fbdfdd94b4ae5969cef: Status 404 returned error can't find the container with id 884ebfbcb654723c19c7802d4c7986cab29f810566449fbdfdd94b4ae5969cef May 28 21:04:33.697525 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.697459 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 21:04:33.780956 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.780924 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvjlv\" (UniqueName: \"kubernetes.io/projected/962b77c6-d65f-4104-ba14-205c1b6e5c57-kube-api-access-jvjlv\") pod \"payload-processing-58b6c8fdc7-r7dwz\" (UID: \"962b77c6-d65f-4104-ba14-205c1b6e5c57\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:33.881999 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.881945 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jvjlv\" (UniqueName: \"kubernetes.io/projected/962b77c6-d65f-4104-ba14-205c1b6e5c57-kube-api-access-jvjlv\") pod \"payload-processing-58b6c8fdc7-r7dwz\" (UID: \"962b77c6-d65f-4104-ba14-205c1b6e5c57\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:33.896623 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.896592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvjlv\" (UniqueName: \"kubernetes.io/projected/962b77c6-d65f-4104-ba14-205c1b6e5c57-kube-api-access-jvjlv\") pod \"payload-processing-58b6c8fdc7-r7dwz\" (UID: \"962b77c6-d65f-4104-ba14-205c1b6e5c57\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:33.995938 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:33.995901 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:34.126906 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:34.126883 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-r7dwz"] May 28 21:04:34.129325 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:04:34.129298 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod962b77c6_d65f_4104_ba14_205c1b6e5c57.slice/crio-ddf461e14479e5de18725d5f57bc72e34307e1a1864d746fc0458bba1a7f61d5 WatchSource:0}: Error finding container ddf461e14479e5de18725d5f57bc72e34307e1a1864d746fc0458bba1a7f61d5: Status 404 returned error can't find the container with id ddf461e14479e5de18725d5f57bc72e34307e1a1864d746fc0458bba1a7f61d5 May 28 21:04:34.176540 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:34.176504 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" event={"ID":"962b77c6-d65f-4104-ba14-205c1b6e5c57","Type":"ContainerStarted","Data":"ddf461e14479e5de18725d5f57bc72e34307e1a1864d746fc0458bba1a7f61d5"} May 28 21:04:34.177396 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:34.177368 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-fc9cf5979-tx2b7" event={"ID":"7809dda2-6bba-4a4f-939c-aecf65f7f892","Type":"ContainerStarted","Data":"884ebfbcb654723c19c7802d4c7986cab29f810566449fbdfdd94b4ae5969cef"} May 28 21:04:36.187411 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.187371 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" event={"ID":"962b77c6-d65f-4104-ba14-205c1b6e5c57","Type":"ContainerStarted","Data":"0f93dec81d072112c1ffb66771bd0ed8e4aa88ba009efe7162448e6f3c513c3a"} May 28 21:04:36.188005 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.187457 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:04:36.188820 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.188788 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-fc9cf5979-tx2b7" event={"ID":"7809dda2-6bba-4a4f-939c-aecf65f7f892","Type":"ContainerStarted","Data":"f7516b042b91f4166a0653b1e84273c7d09179fbb5d0be3100d94dff17a0c84d"} May 28 21:04:36.188933 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.188916 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:36.202375 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.202252 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" podStartSLOduration=1.57283935 podStartE2EDuration="3.20223535s" podCreationTimestamp="2026-05-28 21:04:33 +0000 UTC" firstStartedPulling="2026-05-28 21:04:34.131091485 +0000 UTC m=+761.746926182" lastFinishedPulling="2026-05-28 21:04:35.760487479 +0000 UTC m=+763.376322182" observedRunningTime="2026-05-28 21:04:36.201410559 +0000 UTC m=+763.817245279" watchObservedRunningTime="2026-05-28 21:04:36.20223535 +0000 UTC m=+763.818070070" May 28 21:04:36.215121 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:36.215034 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-fc9cf5979-tx2b7" podStartSLOduration=1.176321525 podStartE2EDuration="3.215016939s" podCreationTimestamp="2026-05-28 21:04:33 +0000 UTC" firstStartedPulling="2026-05-28 21:04:33.697602206 +0000 UTC m=+761.313436904" lastFinishedPulling="2026-05-28 21:04:35.73629762 +0000 UTC m=+763.352132318" observedRunningTime="2026-05-28 21:04:36.21441715 +0000 UTC m=+763.830251869" watchObservedRunningTime="2026-05-28 21:04:36.215016939 +0000 UTC m=+763.830851655" May 28 21:04:42.198678 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:42.198649 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-fc9cf5979-tx2b7" May 28 21:04:47.194601 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:04:47.194571 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-r7dwz" May 28 21:05:27.015653 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.015565 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5"] May 28 21:05:27.018194 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.018172 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.020720 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.020696 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-5qz76\"" May 28 21:05:27.020841 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.020731 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 28 21:05:27.020841 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.020741 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" May 28 21:05:27.020841 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.020764 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 28 21:05:27.032168 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.032147 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5"] May 28 21:05:27.093389 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093353 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc225ca8-e309-42f9-8dd9-d5839e483d6e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.093590 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.093590 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.093714 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.093714 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093635 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh92v\" (UniqueName: \"kubernetes.io/projected/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kube-api-access-dh92v\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.093714 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.093665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194148 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194116 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194331 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194157 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dh92v\" (UniqueName: \"kubernetes.io/projected/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kube-api-access-dh92v\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194331 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194185 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194331 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194260 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc225ca8-e309-42f9-8dd9-d5839e483d6e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194331 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194290 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194544 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194336 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194594 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194707 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194684 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.194769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.194731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.196638 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.196615 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc225ca8-e309-42f9-8dd9-d5839e483d6e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.196994 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.196969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc225ca8-e309-42f9-8dd9-d5839e483d6e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.207347 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.207316 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh92v\" (UniqueName: \"kubernetes.io/projected/fc225ca8-e309-42f9-8dd9-d5839e483d6e-kube-api-access-dh92v\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5\" (UID: \"fc225ca8-e309-42f9-8dd9-d5839e483d6e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.327175 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.327086 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:27.463600 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:27.463566 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5"] May 28 21:05:27.470333 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:05:27.470298 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc225ca8_e309_42f9_8dd9_d5839e483d6e.slice/crio-ec6391c255e83e1e7fcef5680c77f358a91ed53a4ba1028db012fa28fb7a44e3 WatchSource:0}: Error finding container ec6391c255e83e1e7fcef5680c77f358a91ed53a4ba1028db012fa28fb7a44e3: Status 404 returned error can't find the container with id ec6391c255e83e1e7fcef5680c77f358a91ed53a4ba1028db012fa28fb7a44e3 May 28 21:05:28.361877 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:28.361838 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" event={"ID":"fc225ca8-e309-42f9-8dd9-d5839e483d6e","Type":"ContainerStarted","Data":"ec6391c255e83e1e7fcef5680c77f358a91ed53a4ba1028db012fa28fb7a44e3"} May 28 21:05:34.388769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:34.388733 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" event={"ID":"fc225ca8-e309-42f9-8dd9-d5839e483d6e","Type":"ContainerStarted","Data":"e360d38edba06c62d8d93b77bd3bfd1cfe13d22d0ce00207424f9e59fffaa63f"} May 28 21:05:42.420545 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:42.420513 2570 generic.go:358] "Generic (PLEG): container finished" podID="fc225ca8-e309-42f9-8dd9-d5839e483d6e" containerID="e360d38edba06c62d8d93b77bd3bfd1cfe13d22d0ce00207424f9e59fffaa63f" exitCode=0 May 28 21:05:42.420935 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:42.420554 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" event={"ID":"fc225ca8-e309-42f9-8dd9-d5839e483d6e","Type":"ContainerDied","Data":"e360d38edba06c62d8d93b77bd3bfd1cfe13d22d0ce00207424f9e59fffaa63f"} May 28 21:05:44.428593 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:44.428556 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" event={"ID":"fc225ca8-e309-42f9-8dd9-d5839e483d6e","Type":"ContainerStarted","Data":"1a908a893493d9d433806471256afb21a07712e1471563a24fd9864cdf2b09c8"} May 28 21:05:44.429069 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:44.428785 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:44.446645 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:44.446594 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" podStartSLOduration=2.167155783 podStartE2EDuration="18.446580076s" podCreationTimestamp="2026-05-28 21:05:26 +0000 UTC" firstStartedPulling="2026-05-28 21:05:27.472972359 +0000 UTC m=+815.088807058" lastFinishedPulling="2026-05-28 21:05:43.752396649 +0000 UTC m=+831.368231351" observedRunningTime="2026-05-28 21:05:44.446161951 +0000 UTC m=+832.061996670" watchObservedRunningTime="2026-05-28 21:05:44.446580076 +0000 UTC m=+832.062414832" May 28 21:05:55.445144 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:55.445113 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5" May 28 21:05:59.619084 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.619047 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7"] May 28 21:05:59.749399 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.749366 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7"] May 28 21:05:59.749570 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.749505 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.751209 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.751186 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" May 28 21:05:59.789246 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789214 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b0f811-8c7e-4654-af49-6874df8082d6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.789361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789300 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.789361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.789361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789354 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.789498 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789380 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrrt\" (UniqueName: \"kubernetes.io/projected/86b0f811-8c7e-4654-af49-6874df8082d6-kube-api-access-nfrrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.789498 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.789428 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890040 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.889954 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b0f811-8c7e-4654-af49-6874df8082d6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890040 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890026 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890257 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890047 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890257 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890064 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890257 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890094 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrrt\" (UniqueName: \"kubernetes.io/projected/86b0f811-8c7e-4654-af49-6874df8082d6-kube-api-access-nfrrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890257 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890141 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890457 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890404 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890562 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890513 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.890609 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.890580 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.892330 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.892305 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/86b0f811-8c7e-4654-af49-6874df8082d6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.892602 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.892586 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b0f811-8c7e-4654-af49-6874df8082d6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:05:59.897707 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:05:59.897687 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrrt\" (UniqueName: \"kubernetes.io/projected/86b0f811-8c7e-4654-af49-6874df8082d6-kube-api-access-nfrrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7\" (UID: \"86b0f811-8c7e-4654-af49-6874df8082d6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:06:00.060057 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:00.060016 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:06:00.198891 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:00.198870 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7"] May 28 21:06:00.201396 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:06:00.201360 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b0f811_8c7e_4654_af49_6874df8082d6.slice/crio-524514820ad7e58eae9386a93c3a9b385d6e1fdd4fc248a93211aed754486c3a WatchSource:0}: Error finding container 524514820ad7e58eae9386a93c3a9b385d6e1fdd4fc248a93211aed754486c3a: Status 404 returned error can't find the container with id 524514820ad7e58eae9386a93c3a9b385d6e1fdd4fc248a93211aed754486c3a May 28 21:06:00.488110 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:00.488032 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" event={"ID":"86b0f811-8c7e-4654-af49-6874df8082d6","Type":"ContainerStarted","Data":"7eec9b4ef3bdd3d336311d8307805bbdf79df1092d49d0c3e893cd8b6973476d"} May 28 21:06:00.488110 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:00.488069 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" event={"ID":"86b0f811-8c7e-4654-af49-6874df8082d6","Type":"ContainerStarted","Data":"524514820ad7e58eae9386a93c3a9b385d6e1fdd4fc248a93211aed754486c3a"} May 28 21:06:06.511640 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:06.511607 2570 generic.go:358] "Generic (PLEG): container finished" podID="86b0f811-8c7e-4654-af49-6874df8082d6" containerID="7eec9b4ef3bdd3d336311d8307805bbdf79df1092d49d0c3e893cd8b6973476d" exitCode=0 May 28 21:06:06.512008 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:06.511657 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" event={"ID":"86b0f811-8c7e-4654-af49-6874df8082d6","Type":"ContainerDied","Data":"7eec9b4ef3bdd3d336311d8307805bbdf79df1092d49d0c3e893cd8b6973476d"} May 28 21:06:07.516960 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:07.516926 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" event={"ID":"86b0f811-8c7e-4654-af49-6874df8082d6","Type":"ContainerStarted","Data":"6c00d23f49765226de1b5c7b8747013487afb50c7c9471537988f81ca29215c9"} May 28 21:06:07.517315 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:07.517142 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:06:07.538331 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:07.538279 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" podStartSLOduration=8.297277778 podStartE2EDuration="8.538266787s" podCreationTimestamp="2026-05-28 21:05:59 +0000 UTC" firstStartedPulling="2026-05-28 21:06:06.512222453 +0000 UTC m=+854.128057152" lastFinishedPulling="2026-05-28 21:06:06.753211464 +0000 UTC m=+854.369046161" observedRunningTime="2026-05-28 21:06:07.535807777 +0000 UTC m=+855.151642508" watchObservedRunningTime="2026-05-28 21:06:07.538266787 +0000 UTC m=+855.154101506" May 28 21:06:18.534768 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:18.534734 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7" May 28 21:06:40.812175 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.812141 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl"] May 28 21:06:40.814593 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.814573 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.816450 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.816430 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 28 21:06:40.824326 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.824300 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl"] May 28 21:06:40.934001 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.933955 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vx9g\" (UniqueName: \"kubernetes.io/projected/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kube-api-access-4vx9g\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.934180 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.934058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6deb214f-f11f-4e16-8ffa-8c6770d2282d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.934180 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.934099 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.934180 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.934146 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.934294 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.934229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:40.934294 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:40.934273 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035569 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035523 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6deb214f-f11f-4e16-8ffa-8c6770d2282d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035578 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035629 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035670 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035708 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.035760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.035752 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4vx9g\" (UniqueName: \"kubernetes.io/projected/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kube-api-access-4vx9g\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.036148 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.036072 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.036296 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.036167 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.036367 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.036306 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.038056 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.038036 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6deb214f-f11f-4e16-8ffa-8c6770d2282d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.038189 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.038174 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6deb214f-f11f-4e16-8ffa-8c6770d2282d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.042722 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.042702 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vx9g\" (UniqueName: \"kubernetes.io/projected/6deb214f-f11f-4e16-8ffa-8c6770d2282d-kube-api-access-4vx9g\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl\" (UID: \"6deb214f-f11f-4e16-8ffa-8c6770d2282d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.147452 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.147366 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:41.283230 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.283189 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl"] May 28 21:06:41.286513 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:06:41.286467 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6deb214f_f11f_4e16_8ffa_8c6770d2282d.slice/crio-c94860b11c426a39efc3f60203eb82c9da32354c54c0876c24cf61701a3879e2 WatchSource:0}: Error finding container c94860b11c426a39efc3f60203eb82c9da32354c54c0876c24cf61701a3879e2: Status 404 returned error can't find the container with id c94860b11c426a39efc3f60203eb82c9da32354c54c0876c24cf61701a3879e2 May 28 21:06:41.639729 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.639695 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" event={"ID":"6deb214f-f11f-4e16-8ffa-8c6770d2282d","Type":"ContainerStarted","Data":"a64bb312a09e0aed173ee1e77058ad73c1f35e36b512ee1db211fbf3aeb78c02"} May 28 21:06:41.639729 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:41.639729 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" event={"ID":"6deb214f-f11f-4e16-8ffa-8c6770d2282d","Type":"ContainerStarted","Data":"c94860b11c426a39efc3f60203eb82c9da32354c54c0876c24cf61701a3879e2"} May 28 21:06:47.660462 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:47.660428 2570 generic.go:358] "Generic (PLEG): container finished" podID="6deb214f-f11f-4e16-8ffa-8c6770d2282d" containerID="a64bb312a09e0aed173ee1e77058ad73c1f35e36b512ee1db211fbf3aeb78c02" exitCode=0 May 28 21:06:47.660889 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:47.660501 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" event={"ID":"6deb214f-f11f-4e16-8ffa-8c6770d2282d","Type":"ContainerDied","Data":"a64bb312a09e0aed173ee1e77058ad73c1f35e36b512ee1db211fbf3aeb78c02"} May 28 21:06:48.666209 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:48.666166 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" event={"ID":"6deb214f-f11f-4e16-8ffa-8c6770d2282d","Type":"ContainerStarted","Data":"928bebfceaf8efdeaa057180f160b0a23d53b3aebc1d9fab1600a9409d2751fe"} May 28 21:06:48.667010 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:48.666987 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:06:48.684199 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:48.684136 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" podStartSLOduration=8.401782164 podStartE2EDuration="8.684118424s" podCreationTimestamp="2026-05-28 21:06:40 +0000 UTC" firstStartedPulling="2026-05-28 21:06:47.661103614 +0000 UTC m=+895.276938312" lastFinishedPulling="2026-05-28 21:06:47.943439863 +0000 UTC m=+895.559274572" observedRunningTime="2026-05-28 21:06:48.682937922 +0000 UTC m=+896.298772641" watchObservedRunningTime="2026-05-28 21:06:48.684118424 +0000 UTC m=+896.299953146" May 28 21:06:59.685509 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:06:59.685459 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl" May 28 21:08:55.358760 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.358717 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:08:55.359627 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.358944 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-764c88474f-rg8gk" podUID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" containerName="manager" containerID="cri-o://5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8" gracePeriod=30 May 28 21:08:55.607596 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.607574 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:08:55.792132 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.792099 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqvpc\" (UniqueName: \"kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc\") pod \"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f\" (UID: \"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f\") " May 28 21:08:55.794127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.794094 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc" (OuterVolumeSpecName: "kube-api-access-rqvpc") pod "6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" (UID: "6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f"). InnerVolumeSpecName "kube-api-access-rqvpc". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:08:55.893483 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:55.893444 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqvpc\" (UniqueName: \"kubernetes.io/projected/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f-kube-api-access-rqvpc\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 21:08:56.111592 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.111510 2570 generic.go:358] "Generic (PLEG): container finished" podID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" containerID="5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8" exitCode=0 May 28 21:08:56.111592 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.111568 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-rg8gk" May 28 21:08:56.111769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.111595 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-rg8gk" event={"ID":"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f","Type":"ContainerDied","Data":"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8"} May 28 21:08:56.111769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.111628 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-rg8gk" event={"ID":"6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f","Type":"ContainerDied","Data":"dee5a3f076812c35027e90597f26df9e52604d720989f40587aabf4a5ef01839"} May 28 21:08:56.111769 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.111644 2570 scope.go:117] "RemoveContainer" containerID="5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8" May 28 21:08:56.120213 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.120193 2570 scope.go:117] "RemoveContainer" containerID="5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8" May 28 21:08:56.120441 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:08:56.120424 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8\": container with ID starting with 5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8 not found: ID does not exist" containerID="5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8" May 28 21:08:56.120509 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.120447 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8"} err="failed to get container status \"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8\": rpc error: code = NotFound desc = could not find container \"5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8\": container with ID starting with 5cfc3d4ad5b70a8d04234cce31a8f9e4e9e13931aa51b5bb735126214983b4a8 not found: ID does not exist" May 28 21:08:56.130726 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.130701 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:08:56.134721 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.134701 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-764c88474f-rg8gk"] May 28 21:08:56.861460 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.861429 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764c88474f-q5z9c"] May 28 21:08:56.861855 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.861746 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" containerName="manager" May 28 21:08:56.861855 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.861757 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" containerName="manager" May 28 21:08:56.861855 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.861825 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" containerName="manager" May 28 21:08:56.864023 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.864006 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:56.865782 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.865762 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 28 21:08:56.865891 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.865819 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2wqh5\"" May 28 21:08:56.874797 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.874772 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764c88474f-q5z9c"] May 28 21:08:56.904393 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.904343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnmc9\" (UniqueName: \"kubernetes.io/projected/f4fa0b40-5fff-4792-a820-0c06ee569074-kube-api-access-xnmc9\") pod \"maas-controller-764c88474f-q5z9c\" (UID: \"f4fa0b40-5fff-4792-a820-0c06ee569074\") " pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:56.992580 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:56.992551 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f" path="/var/lib/kubelet/pods/6b4c24d2-efe9-4ad5-89a0-4ebd8b9bd00f/volumes" May 28 21:08:57.005810 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:57.005783 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xnmc9\" (UniqueName: \"kubernetes.io/projected/f4fa0b40-5fff-4792-a820-0c06ee569074-kube-api-access-xnmc9\") pod \"maas-controller-764c88474f-q5z9c\" (UID: \"f4fa0b40-5fff-4792-a820-0c06ee569074\") " pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:57.013228 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:57.013192 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnmc9\" (UniqueName: \"kubernetes.io/projected/f4fa0b40-5fff-4792-a820-0c06ee569074-kube-api-access-xnmc9\") pod \"maas-controller-764c88474f-q5z9c\" (UID: \"f4fa0b40-5fff-4792-a820-0c06ee569074\") " pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:57.177170 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:57.177078 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:57.305662 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:57.305622 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764c88474f-q5z9c"] May 28 21:08:57.308581 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:08:57.308541 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4fa0b40_5fff_4792_a820_0c06ee569074.slice/crio-3a120c39b51a3426f5a388cc24973a10362ee741996831afae25f263425f54bb WatchSource:0}: Error finding container 3a120c39b51a3426f5a388cc24973a10362ee741996831afae25f263425f54bb: Status 404 returned error can't find the container with id 3a120c39b51a3426f5a388cc24973a10362ee741996831afae25f263425f54bb May 28 21:08:58.120929 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:58.120890 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-q5z9c" event={"ID":"f4fa0b40-5fff-4792-a820-0c06ee569074","Type":"ContainerStarted","Data":"cc0a5c8738c5914fca939b0d9260199254bbd7469c89cf042d2cbfed42d89a5f"} May 28 21:08:58.120929 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:58.120925 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764c88474f-q5z9c" event={"ID":"f4fa0b40-5fff-4792-a820-0c06ee569074","Type":"ContainerStarted","Data":"3a120c39b51a3426f5a388cc24973a10362ee741996831afae25f263425f54bb"} May 28 21:08:58.121343 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:58.121017 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:08:58.136939 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:08:58.136894 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-764c88474f-q5z9c" podStartSLOduration=1.737493217 podStartE2EDuration="2.136881698s" podCreationTimestamp="2026-05-28 21:08:56 +0000 UTC" firstStartedPulling="2026-05-28 21:08:57.309735452 +0000 UTC m=+1024.925570150" lastFinishedPulling="2026-05-28 21:08:57.709123928 +0000 UTC m=+1025.324958631" observedRunningTime="2026-05-28 21:08:58.134462546 +0000 UTC m=+1025.750297267" watchObservedRunningTime="2026-05-28 21:08:58.136881698 +0000 UTC m=+1025.752716417" May 28 21:09:09.129773 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:09:09.129740 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764c88474f-q5z9c" May 28 21:19:15.726566 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:15.726524 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:19:15.729270 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:15.726792 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" podUID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" containerName="manager" containerID="cri-o://5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78" gracePeriod=10 May 28 21:19:16.174490 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.174454 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:19:16.214904 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.214876 2570 generic.go:358] "Generic (PLEG): container finished" podID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" containerID="5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78" exitCode=0 May 28 21:19:16.215066 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.214964 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" May 28 21:19:16.215066 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.214967 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" event={"ID":"0bdba32b-b267-4b88-af74-51efb4a4bfe1","Type":"ContainerDied","Data":"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78"} May 28 21:19:16.215066 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.215012 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp" event={"ID":"0bdba32b-b267-4b88-af74-51efb4a4bfe1","Type":"ContainerDied","Data":"ee9eb0078ee84dfd96fa3abde3a5b3634e77e17dc65f40e0d08eaefe5d6c8257"} May 28 21:19:16.215066 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.215038 2570 scope.go:117] "RemoveContainer" containerID="5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78" May 28 21:19:16.223361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.223343 2570 scope.go:117] "RemoveContainer" containerID="5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78" May 28 21:19:16.223638 ip-10-0-136-77 kubenswrapper[2570]: E0528 21:19:16.223618 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78\": container with ID starting with 5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78 not found: ID does not exist" containerID="5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78" May 28 21:19:16.223706 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.223646 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78"} err="failed to get container status \"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78\": rpc error: code = NotFound desc = could not find container \"5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78\": container with ID starting with 5e1719abcd909d9e46fd7656beeb9b66aa3f42c004449728d4bbcf56707d5c78 not found: ID does not exist" May 28 21:19:16.237990 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.237965 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw5nx\" (UniqueName: \"kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx\") pod \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " May 28 21:19:16.238106 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.238001 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume\") pod \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\" (UID: \"0bdba32b-b267-4b88-af74-51efb4a4bfe1\") " May 28 21:19:16.238432 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.238411 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0bdba32b-b267-4b88-af74-51efb4a4bfe1" (UID: "0bdba32b-b267-4b88-af74-51efb4a4bfe1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 28 21:19:16.240448 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.240425 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx" (OuterVolumeSpecName: "kube-api-access-hw5nx") pod "0bdba32b-b267-4b88-af74-51efb4a4bfe1" (UID: "0bdba32b-b267-4b88-af74-51efb4a4bfe1"). InnerVolumeSpecName "kube-api-access-hw5nx". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 28 21:19:16.339169 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.339071 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw5nx\" (UniqueName: \"kubernetes.io/projected/0bdba32b-b267-4b88-af74-51efb4a4bfe1-kube-api-access-hw5nx\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 21:19:16.339169 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.339105 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0bdba32b-b267-4b88-af74-51efb4a4bfe1-extensions-socket-volume\") on node \"ip-10-0-136-77.ec2.internal\" DevicePath \"\"" May 28 21:19:16.539199 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.539170 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:19:16.542599 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.542568 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-745mp"] May 28 21:19:16.991682 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:19:16.991651 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" path="/var/lib/kubelet/pods/0bdba32b-b267-4b88-af74-51efb4a4bfe1/volumes" May 28 21:20:21.813064 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.812974 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd"] May 28 21:20:21.813552 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.813449 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" containerName="manager" May 28 21:20:21.813552 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.813496 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" containerName="manager" May 28 21:20:21.813679 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.813583 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="0bdba32b-b267-4b88-af74-51efb4a4bfe1" containerName="manager" May 28 21:20:21.816866 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.816843 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:21.818801 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.818774 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-t4hzt\"" May 28 21:20:21.827339 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.827311 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd"] May 28 21:20:21.896333 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.896297 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptvp9\" (UniqueName: \"kubernetes.io/projected/d994db27-c3bf-4dba-8933-0bcd75c98862-kube-api-access-ptvp9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:21.896536 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.896354 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d994db27-c3bf-4dba-8933-0bcd75c98862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:21.997286 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.997248 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ptvp9\" (UniqueName: \"kubernetes.io/projected/d994db27-c3bf-4dba-8933-0bcd75c98862-kube-api-access-ptvp9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:21.997462 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.997305 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d994db27-c3bf-4dba-8933-0bcd75c98862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:21.997719 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:21.997700 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d994db27-c3bf-4dba-8933-0bcd75c98862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:22.006976 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.006954 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptvp9\" (UniqueName: \"kubernetes.io/projected/d994db27-c3bf-4dba-8933-0bcd75c98862-kube-api-access-ptvp9\") pod \"kuadrant-operator-controller-manager-55c7f4c975-cccnd\" (UID: \"d994db27-c3bf-4dba-8933-0bcd75c98862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:22.126893 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.126811 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:22.265127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.265088 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd"] May 28 21:20:22.270332 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:20:22.270304 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd994db27_c3bf_4dba_8933_0bcd75c98862.slice/crio-bcd9c232521dc967102ae4d95e1e723ecac237f51b05fca84e86488e5e334c0b WatchSource:0}: Error finding container bcd9c232521dc967102ae4d95e1e723ecac237f51b05fca84e86488e5e334c0b: Status 404 returned error can't find the container with id bcd9c232521dc967102ae4d95e1e723ecac237f51b05fca84e86488e5e334c0b May 28 21:20:22.273296 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.273275 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 21:20:22.433295 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.433196 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" event={"ID":"d994db27-c3bf-4dba-8933-0bcd75c98862","Type":"ContainerStarted","Data":"6822525beacd9f916b0a90c364928286dc442ddb0a4a5c8fd4e6822e3f2caac1"} May 28 21:20:22.433295 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.433242 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" event={"ID":"d994db27-c3bf-4dba-8933-0bcd75c98862","Type":"ContainerStarted","Data":"bcd9c232521dc967102ae4d95e1e723ecac237f51b05fca84e86488e5e334c0b"} May 28 21:20:22.433467 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.433328 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:20:22.456380 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:22.456329 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" podStartSLOduration=1.45631336 podStartE2EDuration="1.45631336s" podCreationTimestamp="2026-05-28 21:20:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 21:20:22.455153306 +0000 UTC m=+1710.070988028" watchObservedRunningTime="2026-05-28 21:20:22.45631336 +0000 UTC m=+1710.072148080" May 28 21:20:33.438979 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:20:33.438949 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55c7f4c975-cccnd" May 28 21:30:11.023606 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:11.023573 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-dtlmn_dacdada3-fb15-43b7-9a78-fc447ad75911/manager/0.log" May 28 21:30:11.136610 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:11.136580 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-fc9cf5979-tx2b7_7809dda2-6bba-4a4f-939c-aecf65f7f892/maas-api/0.log" May 28 21:30:11.482854 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:11.482783 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764c88474f-q5z9c_f4fa0b40-5fff-4792-a820-0c06ee569074/manager/0.log" May 28 21:30:11.600032 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:11.600006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7j9td_499ea2a9-9c07-4025-a40c-8ac7686f3101/manager/2.log" May 28 21:30:11.715127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:11.715099 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc97fc4b4-8pznj_d5c11e9a-45ab-4ccf-bc86-d72776d29d60/manager/0.log" May 28 21:30:13.427847 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:13.427815 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-wv8zl_09eb92c6-17f6-4064-9996-fd228ce35b3e/manager/0.log" May 28 21:30:13.875102 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:13.875070 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-cccnd_d994db27-c3bf-4dba-8933-0bcd75c98862/manager/0.log" May 28 21:30:14.537783 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:14.537749 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-d5bh6_36ef0a7c-bf5d-4171-b347-68003c93c5f1/discovery/0.log" May 28 21:30:14.746200 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:14.746150 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-774cfddf4d-r5mn6_6e662cd7-de9c-4e54-8fc2-9993a6397b18/kube-auth-proxy/0.log" May 28 21:30:14.975216 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:14.975126 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-r7dwz_962b77c6-d65f-4104-ba14-205c1b6e5c57/payload-processing/0.log" May 28 21:30:15.749324 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.749295 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7_86b0f811-8c7e-4654-af49-6874df8082d6/storage-initializer/0.log" May 28 21:30:15.756136 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.756111 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccpcmc7_86b0f811-8c7e-4654-af49-6874df8082d6/main/0.log" May 28 21:30:15.865791 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.865761 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl_6deb214f-f11f-4e16-8ffa-8c6770d2282d/storage-initializer/0.log" May 28 21:30:15.871775 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.871752 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-qfmwl_6deb214f-f11f-4e16-8ffa-8c6770d2282d/main/0.log" May 28 21:30:15.974436 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.974410 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5_fc225ca8-e309-42f9-8dd9-d5839e483d6e/storage-initializer/0.log" May 28 21:30:15.980857 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:15.980832 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-gcdd5_fc225ca8-e309-42f9-8dd9-d5839e483d6e/main/0.log" May 28 21:30:22.492325 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:22.492292 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rqdfl_5746890a-97c1-4544-971b-e0d15add347d/global-pull-secret-syncer/0.log" May 28 21:30:22.598612 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:22.598580 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gtpfl_f654f73d-9fd6-4361-b611-d853c545123f/konnectivity-agent/0.log" May 28 21:30:22.669333 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:22.669303 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-77.ec2.internal_18bc8a3cba233c5956614825d898bb27/haproxy/0.log" May 28 21:30:26.861709 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:26.861679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-wv8zl_09eb92c6-17f6-4064-9996-fd228ce35b3e/manager/0.log" May 28 21:30:27.018573 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:27.018543 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-55c7f4c975-cccnd_d994db27-c3bf-4dba-8933-0bcd75c98862/manager/0.log" May 28 21:30:28.703204 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:28.703178 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2jbk_f1985411-2d30-4df7-9e4d-0208bf97dcf4/node-exporter/0.log" May 28 21:30:28.732549 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:28.732529 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2jbk_f1985411-2d30-4df7-9e4d-0208bf97dcf4/kube-rbac-proxy/0.log" May 28 21:30:28.759163 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:28.759144 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n2jbk_f1985411-2d30-4df7-9e4d-0208bf97dcf4/init-textfile/0.log" May 28 21:30:31.081869 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.081836 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf"] May 28 21:30:31.085101 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.085075 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.086816 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.086792 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjxh6\"/\"openshift-service-ca.crt\"" May 28 21:30:31.086933 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.086801 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hjxh6\"/\"kube-root-ca.crt\"" May 28 21:30:31.086933 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.086803 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hjxh6\"/\"default-dockercfg-dbw98\"" May 28 21:30:31.092100 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.091838 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf"] May 28 21:30:31.147194 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.147153 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzmm\" (UniqueName: \"kubernetes.io/projected/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-kube-api-access-7pzmm\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.147194 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.147191 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-podres\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.147407 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.147226 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-sys\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.147407 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.147280 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-lib-modules\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.147407 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.147308 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-proc\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248087 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzmm\" (UniqueName: \"kubernetes.io/projected/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-kube-api-access-7pzmm\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248127 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248134 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-podres\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248177 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-sys\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248202 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-lib-modules\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248219 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-proc\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248259 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-podres\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-sys\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248341 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-lib-modules\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.248361 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.248353 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-proc\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.254846 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.254821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzmm\" (UniqueName: \"kubernetes.io/projected/8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de-kube-api-access-7pzmm\") pod \"perf-node-gather-daemonset-tfbhf\" (UID: \"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de\") " pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.396177 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.396077 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:31.531920 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.531891 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf"] May 28 21:30:31.534702 ip-10-0-136-77 kubenswrapper[2570]: W0528 21:30:31.534667 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8cb2dbcc_c4ae_44a8_a5a4_8d870969d8de.slice/crio-ef9ce7851b6e16a056df912859c7792c0e23611881d3ccd88012e8df7139c1f2 WatchSource:0}: Error finding container ef9ce7851b6e16a056df912859c7792c0e23611881d3ccd88012e8df7139c1f2: Status 404 returned error can't find the container with id ef9ce7851b6e16a056df912859c7792c0e23611881d3ccd88012e8df7139c1f2 May 28 21:30:31.536596 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:31.536578 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 28 21:30:32.501170 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:32.501135 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" event={"ID":"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de","Type":"ContainerStarted","Data":"d2e277b40b51b326b342a2d135791e5ea456b140b985e32215be3cb4ded3c5e2"} May 28 21:30:32.501170 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:32.501170 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" event={"ID":"8cb2dbcc-c4ae-44a8-a5a4-8d870969d8de","Type":"ContainerStarted","Data":"ef9ce7851b6e16a056df912859c7792c0e23611881d3ccd88012e8df7139c1f2"} May 28 21:30:32.501595 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:32.501262 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:32.514207 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:32.514158 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" podStartSLOduration=1.514141728 podStartE2EDuration="1.514141728s" podCreationTimestamp="2026-05-28 21:30:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 21:30:32.512881223 +0000 UTC m=+2320.128715957" watchObservedRunningTime="2026-05-28 21:30:32.514141728 +0000 UTC m=+2320.129976448" May 28 21:30:33.011595 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:33.011566 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2nlt4_fdbfb640-0d41-45f2-9aa8-e3bf40592de5/dns/0.log" May 28 21:30:33.032289 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:33.032265 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2nlt4_fdbfb640-0d41-45f2-9aa8-e3bf40592de5/kube-rbac-proxy/0.log" May 28 21:30:33.172309 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:33.172279 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jstj6_1bed4256-2823-4ff3-a5e1-dabf5429faa7/dns-node-resolver/0.log" May 28 21:30:33.735086 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:33.735047 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hzr92_9d377526-7f51-4275-ad9e-c99a0a1766ed/node-ca/0.log" May 28 21:30:34.698276 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:34.698246 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-d5bh6_36ef0a7c-bf5d-4171-b347-68003c93c5f1/discovery/0.log" May 28 21:30:34.741604 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:34.741580 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-774cfddf4d-r5mn6_6e662cd7-de9c-4e54-8fc2-9993a6397b18/kube-auth-proxy/0.log" May 28 21:30:34.873168 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:34.873142 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-r7dwz_962b77c6-d65f-4104-ba14-205c1b6e5c57/payload-processing/0.log" May 28 21:30:35.455619 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:35.455591 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vqhmn_9b62715f-a93e-4544-8a69-3d443630e2d2/serve-healthcheck-canary/0.log" May 28 21:30:35.993876 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:35.993848 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gw2nm_0f0a4eac-7264-4a7b-9a79-84a9c507ffd7/kube-rbac-proxy/0.log" May 28 21:30:36.014059 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:36.014030 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gw2nm_0f0a4eac-7264-4a7b-9a79-84a9c507ffd7/exporter/0.log" May 28 21:30:36.034183 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:36.034157 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gw2nm_0f0a4eac-7264-4a7b-9a79-84a9c507ffd7/extractor/0.log" May 28 21:30:37.918388 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:37.918360 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-dtlmn_dacdada3-fb15-43b7-9a78-fc447ad75911/manager/0.log" May 28 21:30:37.950848 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:37.950824 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-fc9cf5979-tx2b7_7809dda2-6bba-4a4f-939c-aecf65f7f892/maas-api/0.log" May 28 21:30:38.066893 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:38.066861 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764c88474f-q5z9c_f4fa0b40-5fff-4792-a820-0c06ee569074/manager/0.log" May 28 21:30:38.089191 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:38.089168 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7j9td_499ea2a9-9c07-4025-a40c-8ac7686f3101/manager/1.log" May 28 21:30:38.100063 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:38.100032 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-7j9td_499ea2a9-9c07-4025-a40c-8ac7686f3101/manager/2.log" May 28 21:30:38.121206 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:38.121186 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6dc97fc4b4-8pznj_d5c11e9a-45ab-4ccf-bc86-d72776d29d60/manager/0.log" May 28 21:30:38.514037 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:38.514013 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hjxh6/perf-node-gather-daemonset-tfbhf" May 28 21:30:39.455371 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:39.455344 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-gp8cx_a97ba9aa-4fa4-45e2-ad8a-293d5ed3f3fb/openshift-lws-operator/0.log" May 28 21:30:45.362661 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.362621 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/kube-multus-additional-cni-plugins/0.log" May 28 21:30:45.383394 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.383370 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/egress-router-binary-copy/0.log" May 28 21:30:45.404338 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.404310 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/cni-plugins/0.log" May 28 21:30:45.424866 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.424847 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/bond-cni-plugin/0.log" May 28 21:30:45.447594 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.447570 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/routeoverride-cni/0.log" May 28 21:30:45.469232 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.469212 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/whereabouts-cni-bincopy/0.log" May 28 21:30:45.489150 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.489127 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p79nd_2d2d247b-766d-4b6f-845f-0d4a22141dc7/whereabouts-cni/0.log" May 28 21:30:45.672604 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.672515 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bxvvb_52d66fbd-189a-41e1-a106-06795d3fe62c/kube-multus/0.log" May 28 21:30:45.792202 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.792174 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mspmh_df288737-cf68-4b61-9f6b-4a87f5df1a98/network-metrics-daemon/0.log" May 28 21:30:45.811374 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:45.811348 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mspmh_df288737-cf68-4b61-9f6b-4a87f5df1a98/kube-rbac-proxy/0.log" May 28 21:30:46.712828 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.712753 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/ovn-controller/0.log" May 28 21:30:46.741959 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.741920 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/ovn-acl-logging/0.log" May 28 21:30:46.762812 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.762779 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/kube-rbac-proxy-node/0.log" May 28 21:30:46.783087 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.783064 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/kube-rbac-proxy-ovn-metrics/0.log" May 28 21:30:46.800893 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.800869 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/northd/0.log" May 28 21:30:46.821204 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.821171 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/nbdb/0.log" May 28 21:30:46.840943 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.840921 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/sbdb/0.log" May 28 21:30:46.930170 ip-10-0-136-77 kubenswrapper[2570]: I0528 21:30:46.930137 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-j8294_8c8228ec-5064-45e2-bee8-aaa03fe0691c/ovnkube-controller/0.log"