Jun 09 11:04:40.357137 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 09 11:04:40.357145 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 09 11:04:40.357152 ip-10-0-130-8 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 09 11:04:40.357294 ip-10-0-130-8 systemd[1]: Failed to start Kubernetes Kubelet. Jun 09 11:04:50.592585 ip-10-0-130-8 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 09 11:04:50.592601 ip-10-0-130-8 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5ab3867fe6574b92bd045089a01be653 -- Jun 09 11:07:41.051897 ip-10-0-130-8 systemd[1]: Starting Kubernetes Kubelet... Jun 09 11:07:41.490538 ip-10-0-130-8 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:07:41.490538 ip-10-0-130-8 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 11:07:41.490538 ip-10-0-130-8 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:07:41.490538 ip-10-0-130-8 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 11:07:41.490538 ip-10-0-130-8 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 11:07:41.491129 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.490626 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 11:07:41.495645 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495631 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.495645 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495644 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495648 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495652 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495665 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495669 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495671 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495675 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495679 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495682 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495685 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495696 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495699 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495702 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495704 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495707 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495710 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495712 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495715 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.495710 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495718 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495721 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495724 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495726 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495729 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495732 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495735 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495737 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495741 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495744 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495746 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495749 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495751 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495753 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495756 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495758 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495761 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495763 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495766 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.496119 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495768 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495770 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495773 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495775 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495778 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495786 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495789 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495791 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495793 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495796 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495798 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495801 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495803 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495806 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495808 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495811 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495814 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495816 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495819 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.496590 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495821 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495824 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495826 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495829 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495831 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495834 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495836 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495839 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495841 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495844 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495846 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495849 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495852 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495854 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495857 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495859 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495861 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495864 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495866 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495878 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.497042 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495881 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495884 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495886 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495889 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495891 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495894 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495897 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495900 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495902 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495905 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495908 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495912 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495915 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495918 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495921 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495923 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495926 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495928 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495931 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495933 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.497602 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495936 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.495939 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496677 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496684 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496688 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496691 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496699 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496702 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496705 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496708 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496710 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496713 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496721 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496724 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496726 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496729 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496731 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496734 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496736 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496739 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.498319 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496742 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496744 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496747 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496749 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496752 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496754 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496757 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496759 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496762 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496764 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496767 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496769 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496772 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496774 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496778 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496782 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496785 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496788 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496790 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496793 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.499086 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496795 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496798 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496800 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496803 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496805 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496815 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496818 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496820 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496823 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496825 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496827 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496830 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496832 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496835 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496837 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496840 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496843 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496845 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496848 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496851 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.499962 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496854 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496856 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496859 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496861 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496863 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496866 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496868 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496871 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496873 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496876 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496878 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496881 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496884 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496886 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496889 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496892 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496894 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496896 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496905 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496908 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.500810 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496910 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496912 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496915 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496918 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496920 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496923 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496926 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496929 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496932 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496934 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496937 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496939 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496942 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496944 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496947 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496949 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496952 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496954 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496957 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.501419 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496960 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.496964 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499083 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499099 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499104 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499110 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499114 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499118 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499124 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499129 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499133 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499137 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499141 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499146 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499150 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499154 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499158 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499162 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499166 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.501982 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499170 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499174 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499178 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499182 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499186 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499190 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499195 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499199 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499203 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499209 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499213 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499217 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499221 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499225 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499229 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499233 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499238 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499242 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499246 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499251 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.502554 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499255 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499259 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499263 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499267 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499271 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499275 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499279 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499283 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499287 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499291 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499295 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499299 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499304 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499307 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499312 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499316 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499320 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499326 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499332 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499336 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.503139 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499341 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499345 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499350 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499355 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499358 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499364 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499370 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499374 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499379 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499384 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499388 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499392 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499396 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499400 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499406 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499413 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499418 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499423 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499427 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.503669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499432 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499436 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499441 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499446 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499450 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499454 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499458 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499463 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499467 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499472 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499476 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499483 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499487 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499491 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499495 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499500 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499504 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499510 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499516 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.504209 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499521 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499525 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499530 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.499535 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499662 2560 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499673 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499687 2560 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499694 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499700 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499705 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499712 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499718 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499723 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499728 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499733 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499738 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499743 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499747 2560 flags.go:64] FLAG: --cgroup-root="" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499752 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499756 2560 flags.go:64] FLAG: --client-ca-file="" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499760 2560 flags.go:64] FLAG: --cloud-provider="external" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499766 2560 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499773 2560 flags.go:64] FLAG: --cluster-domain="" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499778 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 11:07:41.504681 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499783 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499788 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499793 2560 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499798 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499803 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499808 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499814 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499819 2560 flags.go:64] FLAG: --contention-profiling="false" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499824 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499828 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499833 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499838 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499844 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499849 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499854 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499858 2560 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499863 2560 flags.go:64] FLAG: --enable-server="true" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499868 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499876 2560 flags.go:64] FLAG: --event-burst="100" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499881 2560 flags.go:64] FLAG: --event-qps="50" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499886 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499891 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499895 2560 flags.go:64] FLAG: --eviction-hard="" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499901 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 11:07:41.505250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499905 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499910 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499914 2560 flags.go:64] FLAG: --eviction-soft="" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499919 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499924 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499928 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499934 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499938 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499943 2560 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499947 2560 flags.go:64] FLAG: --feature-gates="" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499953 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499957 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499962 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499966 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499971 2560 flags.go:64] FLAG: --healthz-port="10248" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499976 2560 flags.go:64] FLAG: --help="false" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499980 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499985 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499990 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.499995 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500000 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500006 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500010 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 11:07:41.505858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500015 2560 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500019 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500024 2560 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500028 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500033 2560 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500038 2560 flags.go:64] FLAG: --kube-reserved="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500042 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500046 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500051 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500056 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500060 2560 flags.go:64] FLAG: --lock-file="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500065 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500069 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500074 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500082 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500087 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500092 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500097 2560 flags.go:64] FLAG: --logging-format="text" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500101 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500107 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500111 2560 flags.go:64] FLAG: --manifest-url="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500115 2560 flags.go:64] FLAG: --manifest-url-header="" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500121 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500126 2560 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500132 2560 flags.go:64] FLAG: --max-pods="110" Jun 09 11:07:41.506457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500137 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500142 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500147 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500151 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500156 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500160 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500165 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500176 2560 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500180 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500185 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500190 2560 flags.go:64] FLAG: --pod-cidr="" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500194 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500202 2560 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500207 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500212 2560 flags.go:64] FLAG: --pods-per-core="0" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500216 2560 flags.go:64] FLAG: --port="10250" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500221 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500226 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b04215192bb6acdc" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500231 2560 flags.go:64] FLAG: --qos-reserved="" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500235 2560 flags.go:64] FLAG: --read-only-port="10255" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500240 2560 flags.go:64] FLAG: --register-node="true" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500245 2560 flags.go:64] FLAG: --register-with-taints="" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500250 2560 flags.go:64] FLAG: --registry-burst="10" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500255 2560 flags.go:64] FLAG: --registry-qps="5" Jun 09 11:07:41.507125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500259 2560 flags.go:64] FLAG: --reserved-cpus="" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500264 2560 flags.go:64] FLAG: --reserved-memory="" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500270 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500274 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500278 2560 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500283 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500288 2560 flags.go:64] FLAG: --runonce="false" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500293 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500298 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500302 2560 flags.go:64] FLAG: --seccomp-default="false" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500307 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500312 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500316 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500321 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500325 2560 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500330 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500334 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500339 2560 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500343 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500348 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500352 2560 flags.go:64] FLAG: --system-cgroups="" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500357 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500367 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500371 2560 flags.go:64] FLAG: --tls-cert-file="" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500376 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 11:07:41.507735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500382 2560 flags.go:64] FLAG: --tls-min-version="" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500386 2560 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500390 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500395 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500399 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500403 2560 flags.go:64] FLAG: --v="2" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500410 2560 flags.go:64] FLAG: --version="false" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500416 2560 flags.go:64] FLAG: --vmodule="" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500423 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.500428 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500627 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500634 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500638 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500643 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500647 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500651 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500655 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500660 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500664 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500669 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500673 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500677 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.508326 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500681 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500685 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500689 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500693 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500697 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500701 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500705 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500709 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500713 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500717 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500724 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500728 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500732 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500736 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500740 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500744 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500748 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500752 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500756 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500761 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.508861 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500765 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500769 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500773 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500777 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500782 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500785 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500789 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500794 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500797 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500802 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500806 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500810 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500813 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500817 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500821 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500824 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500828 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500832 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500836 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500840 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.509323 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500844 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500848 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500854 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500859 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500865 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500871 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500877 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500883 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500889 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500894 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500898 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500903 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500909 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500914 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500919 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500924 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500928 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500933 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500937 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.509882 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500941 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500945 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500949 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500954 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500958 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500962 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500966 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500969 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500973 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500977 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500982 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500986 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500990 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500994 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.500998 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501004 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501008 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501013 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501016 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501020 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.510417 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501025 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501031 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501036 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501040 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501045 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501049 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501054 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.501058 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.501681 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.508266 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.508361 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.509236 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.509245 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509312 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.510952 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509317 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509320 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509323 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509326 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509329 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509332 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509335 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509338 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509340 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509343 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509346 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509348 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509351 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509353 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509356 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509359 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509361 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509364 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509366 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509369 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.511294 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509372 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509376 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509379 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509382 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509385 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509387 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509391 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509394 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509398 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509400 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509404 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509407 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509411 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509414 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509416 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509419 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509421 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509424 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509427 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.511835 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509430 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509432 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509434 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509437 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509439 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509442 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509444 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509447 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509450 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509452 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509454 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509457 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509459 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509462 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509464 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509467 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509469 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509473 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509476 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.512272 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509479 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509482 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509486 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509489 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509492 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509495 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509498 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509500 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509503 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509505 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509508 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509510 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509513 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509515 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509518 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509521 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509523 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509526 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509528 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509531 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.512724 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509534 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509536 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509538 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509541 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509544 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509546 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509549 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509551 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509554 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509557 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509559 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509575 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509578 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509580 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509584 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509587 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509590 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509593 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509595 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.513181 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509598 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.509603 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509724 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509728 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509731 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509733 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509736 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509739 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509741 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509744 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509746 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509748 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509751 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509754 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 11:07:41.513642 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509756 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509759 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509761 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509764 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509766 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509769 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509771 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509774 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509776 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509779 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509781 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509784 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509786 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509789 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509792 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509795 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509799 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509801 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509804 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509806 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 11:07:41.513967 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509809 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509811 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509813 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509816 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509818 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509821 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509823 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509827 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509830 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509833 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509836 2560 feature_gate.go:328] unrecognized feature gate: Example Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509838 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509841 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509844 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509847 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509850 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509852 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509854 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509857 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509859 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 11:07:41.514424 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509862 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509865 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509867 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509869 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509872 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509875 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509877 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509880 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509882 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509885 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509888 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509891 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509894 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509896 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509899 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509901 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509905 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509908 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509911 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 11:07:41.514908 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509914 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509916 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509919 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509921 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509924 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509927 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509929 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509932 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509934 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509937 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509939 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509942 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509944 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509946 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509949 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509951 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509954 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509956 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509959 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509961 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 11:07:41.515353 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509964 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509967 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509969 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509972 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509974 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509977 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509979 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:41.509982 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.509986 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.510100 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.512042 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.513444 2560 server.go:1019] "Starting client certificate rotation" Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.513547 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 11:07:41.515840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.513588 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 11:07:41.517552 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.517535 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 11:07:41.521525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.521509 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 11:07:41.534222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.534203 2560 log.go:25] "Validated CRI v1 runtime API" Jun 09 11:07:41.536520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.536503 2560 log.go:25] "Validated CRI v1 image API" Jun 09 11:07:41.537329 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.537316 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 11:07:41.540171 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.540151 2560 fs.go:135] Filesystem UUIDs: map[76c7685d-9ccc-491a-8ac9-6f73e6ee2b4f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 82073c66-16cc-451b-bef0-30d99a13c611:/dev/nvme0n1p3] Jun 09 11:07:41.540235 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.540169 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 11:07:41.543011 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.542993 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 11:07:41.545876 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.545771 2560 manager.go:216] Machine: {Timestamp:2026-06-09 11:07:41.545313769 +0000 UTC m=+0.362003565 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196763 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b4fcf19f70aa392f7d50c4b33e183 SystemUUID:ec2b4fcf-19f7-0aa3-92f7-d50c4b33e183 BootID:5ab3867f-e657-4b92-bd04-5089a01be653 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:93:7a:b1:dd:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:93:7a:b1:dd:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:33:db:42:a5:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 11:07:41.545876 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.545870 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 11:07:41.545985 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.545959 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 11:07:41.549671 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.549638 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 11:07:41.549797 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.549674 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-8.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 11:07:41.551072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.551061 2560 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 11:07:41.551108 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.551074 2560 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 11:07:41.551108 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.551081 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 11:07:41.551722 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.551711 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 11:07:41.552582 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.552573 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 11:07:41.555961 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.555950 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 09 11:07:41.556071 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556062 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 11:07:41.556104 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556091 2560 kubelet.go:486] "Attempting to sync node with API server" Jun 09 11:07:41.556136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556104 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 11:07:41.556136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556122 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 11:07:41.556136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556134 2560 kubelet.go:398] "Adding apiserver pod source" Jun 09 11:07:41.556250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556145 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 11:07:41.556526 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556515 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 11:07:41.556576 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.556532 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 11:07:41.558966 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.558950 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 11:07:41.560638 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.560620 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 11:07:41.560702 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.560667 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 11:07:41.561809 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561792 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 11:07:41.561857 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561825 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 11:07:41.561857 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561840 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 11:07:41.561857 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561851 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561868 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561883 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561894 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561899 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561905 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561911 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561919 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 11:07:41.561937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.561931 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 11:07:41.562711 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.562700 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 11:07:41.562750 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.562712 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 11:07:41.566012 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.565990 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 11:07:41.566140 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.566124 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-8.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 11:07:41.566191 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.566152 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 11:07:41.566327 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.566316 2560 server.go:1268] "Started kubelet" Jun 09 11:07:41.566501 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.566475 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 11:07:41.566680 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.566515 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 11:07:41.566743 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.566703 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 11:07:41.567205 ip-10-0-130-8 systemd[1]: Started Kubernetes Kubelet. Jun 09 11:07:41.568317 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.568298 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 11:07:41.570891 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.570866 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-drlxz" Jun 09 11:07:41.571923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.571908 2560 server.go:310] "Adding debug handlers to kubelet server" Jun 09 11:07:41.573298 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.573279 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 11:07:41.574377 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.573191 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-8.ec2.internal.18b76507bd7892ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-8.ec2.internal,UID:ip-10-0-130-8.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-8.ec2.internal,},FirstTimestamp:2026-06-09 11:07:41.566292686 +0000 UTC m=+0.382982482,LastTimestamp:2026-06-09 11:07:41.566292686 +0000 UTC m=+0.382982482,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-8.ec2.internal,}" Jun 09 11:07:41.575929 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.575915 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 11:07:41.575981 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.575933 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 11:07:41.576086 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.576066 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:41.576663 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.576641 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-drlxz" Jun 09 11:07:41.576902 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.576841 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 11:07:41.577123 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577107 2560 factory.go:153] Registering CRI-O factory Jun 09 11:07:41.577179 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577126 2560 factory.go:223] Registration of the crio container factory successfully Jun 09 11:07:41.577222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577179 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 11:07:41.577222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577188 2560 factory.go:55] Registering systemd factory Jun 09 11:07:41.577222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577196 2560 factory.go:223] Registration of the systemd container factory successfully Jun 09 11:07:41.577222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577215 2560 factory.go:103] Registering Raw factory Jun 09 11:07:41.577374 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577228 2560 manager.go:1196] Started watching for new ooms in manager Jun 09 11:07:41.577444 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577430 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 11:07:41.577505 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577498 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 11:07:41.577650 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577635 2560 manager.go:349] Starting recovery of all containers Jun 09 11:07:41.577767 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577755 2560 reconstruct.go:97] "Volume reconstruction finished" Jun 09 11:07:41.577833 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.577826 2560 reconciler.go:29] "Reconciler: start to sync state" Jun 09 11:07:41.582999 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.582971 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-8.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 09 11:07:41.583096 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.583078 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 09 11:07:41.585821 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.585805 2560 manager.go:354] Recovery completed Jun 09 11:07:41.587347 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.587320 2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 09 11:07:41.590017 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.590004 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.591847 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.591834 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.591902 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.591860 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.591902 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.591869 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.592304 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.592289 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 11:07:41.592304 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.592296 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.592304 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.592300 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 11:07:41.592409 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.592313 2560 state_mem.go:36] "Initialized new in-memory state store" Jun 09 11:07:41.592989 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.592975 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.593026 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.593001 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.593026 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.593012 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.593475 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.593464 2560 policy_none.go:49] "None policy: Start" Jun 09 11:07:41.593507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.593479 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 11:07:41.593507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.593489 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 11:07:41.594405 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.594394 2560 policy_none.go:47] "Start" Jun 09 11:07:41.636126 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636113 2560 manager.go:337] "Starting Device Plugin manager" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.636168 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636180 2560 server.go:87] "Starting device plugin registration server" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636407 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636418 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636504 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636590 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.636599 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.637268 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.637317 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.644236 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.645333 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.645369 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.645388 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 11:07:41.647261 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.645422 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 11:07:41.648003 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.647382 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:07:41.737405 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.737386 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.738248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.738236 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.738308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.738261 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.738308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.738270 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.738308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.738289 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.746075 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.746041 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal"] Jun 09 11:07:41.746131 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.746087 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.747287 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.747274 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.747350 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.747299 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.747350 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.747312 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.747584 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.747558 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.747624 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.747591 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-8.ec2.internal\": node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:41.748428 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.748416 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.748594 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.748582 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.748630 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.748607 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.749005 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.748992 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.749064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.749007 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.749064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.749015 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.749064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.749028 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.749064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.749038 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.749176 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.749082 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.750344 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.750328 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.750406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.750359 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 11:07:41.751006 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.750991 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientMemory" Jun 09 11:07:41.751109 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.751016 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 11:07:41.751109 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.751030 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeHasSufficientPID" Jun 09 11:07:41.765884 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.765864 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:41.775038 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.775020 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-8.ec2.internal\" not found" node="ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.778701 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.778683 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.778787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.778706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.778787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.778728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79ba099ecd0a9bcdc399892bd5724baa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"79ba099ecd0a9bcdc399892bd5724baa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.778902 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.778890 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-8.ec2.internal\" not found" node="ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.866463 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.866436 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:41.879289 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.879384 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879296 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.879384 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879321 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79ba099ecd0a9bcdc399892bd5724baa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"79ba099ecd0a9bcdc399892bd5724baa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.879384 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879369 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.879498 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879382 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80a615a479690b22c773ee8338c8ef75-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal\" (UID: \"80a615a479690b22c773ee8338c8ef75\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.879498 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:41.879377 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79ba099ecd0a9bcdc399892bd5724baa-config\") pod \"kube-apiserver-proxy-ip-10-0-130-8.ec2.internal\" (UID: \"79ba099ecd0a9bcdc399892bd5724baa\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:41.967433 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:41.967406 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.068130 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.068109 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.079483 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.079460 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:42.083871 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.083856 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:42.168441 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.168415 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.268938 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.268916 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.369410 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.369363 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.470031 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.470015 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.512560 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.512526 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 11:07:42.513308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.512670 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 11:07:42.570618 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.570597 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.583696 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.583665 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 11:02:41 +0000 UTC" deadline="2027-11-28 18:17:20.060581039 +0000 UTC" Jun 09 11:07:42.583696 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.583695 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12895h9m37.476889468s" Jun 09 11:07:42.592980 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:42.592954 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ba099ecd0a9bcdc399892bd5724baa.slice/crio-0f4a6c311b140ed86df626d7ea54977549997af6fb2a7e1b89be0f28a951307e WatchSource:0}: Error finding container 0f4a6c311b140ed86df626d7ea54977549997af6fb2a7e1b89be0f28a951307e: Status 404 returned error can't find the container with id 0f4a6c311b140ed86df626d7ea54977549997af6fb2a7e1b89be0f28a951307e Jun 09 11:07:42.593395 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:42.593376 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a615a479690b22c773ee8338c8ef75.slice/crio-3dd0d009bb43ddcfdd01842ca8aab98d38dec2804712c17199f6f40d19862b7e WatchSource:0}: Error finding container 3dd0d009bb43ddcfdd01842ca8aab98d38dec2804712c17199f6f40d19862b7e: Status 404 returned error can't find the container with id 3dd0d009bb43ddcfdd01842ca8aab98d38dec2804712c17199f6f40d19862b7e Jun 09 11:07:42.599363 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.599351 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 11:07:42.610946 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.610928 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:07:42.647963 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.647895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" event={"ID":"79ba099ecd0a9bcdc399892bd5724baa","Type":"ContainerStarted","Data":"0f4a6c311b140ed86df626d7ea54977549997af6fb2a7e1b89be0f28a951307e"} Jun 09 11:07:42.648857 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.648835 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"80a615a479690b22c773ee8338c8ef75","Type":"ContainerStarted","Data":"3dd0d009bb43ddcfdd01842ca8aab98d38dec2804712c17199f6f40d19862b7e"} Jun 09 11:07:42.670991 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:42.670972 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-8.ec2.internal\" not found" Jun 09 11:07:42.728390 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.728372 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:07:42.754449 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.754429 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:07:42.776400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.776383 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" Jun 09 11:07:42.792482 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.792464 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 11:07:42.793631 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.793620 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" Jun 09 11:07:42.803294 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:42.803277 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 11:07:43.557346 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.557316 2560 apiserver.go:52] "Watching apiserver" Jun 09 11:07:43.568738 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.568514 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 11:07:43.568945 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.568922 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal","openshift-multus/multus-additional-cni-plugins-9hx4b","openshift-multus/multus-qcdcv","openshift-network-diagnostics/network-check-target-h928v","openshift-network-operator/iptables-alerter-jlt76","kube-system/konnectivity-agent-zhcbk","kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal","openshift-dns/node-resolver-tlcwk","openshift-image-registry/node-ca-jmkhh","openshift-multus/network-metrics-daemon-t6fzv","openshift-ovn-kubernetes/ovnkube-node-78k77","kube-system/global-pull-secret-syncer-d2lzb","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc","openshift-cluster-node-tuning-operator/tuned-gcvw4"] Jun 09 11:07:43.570347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.570331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.570414 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.570393 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:43.571628 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.571605 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.573079 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.573062 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574307 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574320 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574341 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574434 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hsrk4\"" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574534 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 11:07:43.574544 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574539 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.574876 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.574757 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.575427 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.575378 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kwgd9\"" Jun 09 11:07:43.575509 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.575415 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 11:07:43.576298 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.576114 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.576298 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.576265 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 11:07:43.577102 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.577083 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.577310 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.577290 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.577398 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.577318 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.577398 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.577363 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xmbns\"" Jun 09 11:07:43.578423 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.578405 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.578557 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.578542 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.578677 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.578651 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.578812 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.578774 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 11:07:43.579179 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.579128 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-56zzj\"" Jun 09 11:07:43.579750 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.579731 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-htb2h\"" Jun 09 11:07:43.579855 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.579761 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 11:07:43.580144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.580126 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:43.580237 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.580199 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:43.580352 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.580336 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 11:07:43.580903 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.580883 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.582042 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582023 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.582152 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582029 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 11:07:43.582152 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582033 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 11:07:43.582152 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9mwn5\"" Jun 09 11:07:43.582312 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582291 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 11:07:43.582370 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582347 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 11:07:43.582821 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.582807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.584109 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.584084 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.585213 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.585194 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s2d7b\"" Jun 09 11:07:43.585373 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.585330 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 11:07:43.585965 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.585655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.585965 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.585737 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:43.585965 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.585762 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.586202 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.586184 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.586607 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.586587 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l8g72\"" Jun 09 11:07:43.586688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.586616 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.586740 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.586698 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.587391 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.587363 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.588549 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-cni-binary-copy\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.588549 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-multus\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.588696 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-registration-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.588696 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588658 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcth\" (UniqueName: \"kubernetes.io/projected/782be745-9f10-411b-bb45-0f580202e2db-kube-api-access-ftcth\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.588797 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tjwz\" (UniqueName: \"kubernetes.io/projected/75369ceb-252a-4d5e-b532-d20e91e04137-kube-api-access-5tjwz\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.588848 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-multus-certs\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.588900 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-etc-kubernetes\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.588900 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-lib-modules\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.588991 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588931 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cnibin\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.589045 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.588993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-os-release\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.589045 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-system-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589137 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-cnibin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589137 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-conf-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589229 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-systemd\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.589229 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589198 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.589318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589261 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.589318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589305 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.589410 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-socket-dir-parent\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589499 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589480 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-k8s-cni-cncf-io\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589616 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589524 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-netns\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.589616 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.589553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-var-lib-kubelet\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.590122 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590102 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 11:07:43.590740 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-tuned\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.590829 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.590887 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.590887 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.590977 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-run\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.590977 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.591064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.590987 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-sys-fs\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.591064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cd805-64d9-4f13-9360-8461bb56d5b2-konnectivity-ca\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.591161 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-hostroot\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.591161 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-multus-daemon-config\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.591161 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-modprobe-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591284 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-kubernetes\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591284 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591217 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-tmp\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591284 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl2hb\" (UniqueName: \"kubernetes.io/projected/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-kube-api-access-bl2hb\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591415 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff9cd805-64d9-4f13-9360-8461bb56d5b2-agent-certs\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.591415 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591347 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.591415 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591373 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtxh6\" (UniqueName: \"kubernetes.io/projected/f2452907-e6ac-44a0-9f6b-d647c66dee6e-kube-api-access-jtxh6\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.591415 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591399 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-os-release\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.591601 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-kubelet\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.591688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwb74\" (UniqueName: \"kubernetes.io/projected/27d45ca1-bef0-4f44-985b-679862b9856d-kube-api-access-jwb74\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.591754 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysconfig\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-conf\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.591826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591803 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.591941 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591854 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-bin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.592011 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591974 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 11:07:43.592066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.592019 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jl499\"" Jun 09 11:07:43.592066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.591949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-sys\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.592162 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.592068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-host\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.592162 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.592127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-socket-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.592251 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.592161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-device-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.592251 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.592228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.593830 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.593810 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 11:07:43.595959 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.595941 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 11:07:43.620155 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.620136 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ht5mp" Jun 09 11:07:43.628733 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.628716 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ht5mp" Jun 09 11:07:43.677954 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.677930 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 11:07:43.692348 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-netd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.692455 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437089ae-aeec-4a9f-a4af-2c1237cb1357-serviceca\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.692455 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692390 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-iptables-alerter-script\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.692455 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tjwz\" (UniqueName: \"kubernetes.io/projected/75369ceb-252a-4d5e-b532-d20e91e04137-kube-api-access-5tjwz\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.692455 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692446 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-multus-certs\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-etc-kubernetes\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-lib-modules\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692527 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692548 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-multus-certs\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-env-overrides\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692603 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-etc-kubernetes\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-cnibin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-conf-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.692682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692686 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-cnibin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692685 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-lib-modules\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-var-lib-kubelet\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692723 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-conf-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692736 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-var-lib-kubelet\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl2hb\" (UniqueName: \"kubernetes.io/projected/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-kube-api-access-bl2hb\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-k8s-cni-cncf-io\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692822 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-netns\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-sys-fs\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692858 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-k8s-cni-cncf-io\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-netns\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-log-socket\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692926 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-host-slash\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692944 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-run-netns\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692949 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-sys-fs\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692949 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.692983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.693144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693014 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-systemd-units\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drcmw\" (UniqueName: \"kubernetes.io/projected/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-kube-api-access-drcmw\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693056 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0808476-c7a5-4036-b960-6ed233a8e75a-tmp-dir\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-modprobe-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693114 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-tmp\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-etc-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtxh6\" (UniqueName: \"kubernetes.io/projected/f2452907-e6ac-44a0-9f6b-d647c66dee6e-kube-api-access-jtxh6\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693163 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-kubelet\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693181 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysconfig\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-conf\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-sys\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-kubelet\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693243 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693324 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysconfig\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-var-lib-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.693899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-sys\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-ovn\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693406 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-etc-selinux\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693423 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-sysctl-conf\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693427 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-multus\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcth\" (UniqueName: \"kubernetes.io/projected/782be745-9f10-411b-bb45-0f580202e2db-kube-api-access-ftcth\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693553 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-multus\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693597 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693594 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693626 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-dbus\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-systemd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-node-log\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0808476-c7a5-4036-b960-6ed233a8e75a-hosts-file\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693726 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-modprobe-d\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.694448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cnibin\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693856 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-os-release\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693856 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cnibin\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.693896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-system-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694001 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-os-release\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694023 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-system-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694048 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-systemd\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694084 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-tuned\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694095 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-systemd\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff9cd805-64d9-4f13-9360-8461bb56d5b2-agent-certs\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-script-lib\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7xw5\" (UniqueName: \"kubernetes.io/projected/f6d5f375-8e25-4cad-8288-b127c8563011-kube-api-access-k7xw5\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-socket-dir-parent\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cd805-64d9-4f13-9360-8461bb56d5b2-konnectivity-ca\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.695214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694268 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vslpb\" (UniqueName: \"kubernetes.io/projected/437089ae-aeec-4a9f-a4af-2c1237cb1357-kube-api-access-vslpb\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694350 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694369 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-run\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2452907-e6ac-44a0-9f6b-d647c66dee6e-system-cni-dir\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437089ae-aeec-4a9f-a4af-2c1237cb1357-host\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694411 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-socket-dir-parent\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694513 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-hostroot\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-multus-daemon-config\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-kubernetes\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694628 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-config\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694670 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-kubelet-config\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-os-release\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwb74\" (UniqueName: \"kubernetes.io/projected/27d45ca1-bef0-4f44-985b-679862b9856d-kube-api-access-jwb74\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-host\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-socket-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.695973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-kubelet\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-slash\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-bin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694949 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-host-var-lib-cni-bin\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.694986 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-hostroot\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ff9cd805-64d9-4f13-9360-8461bb56d5b2-konnectivity-ca\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695159 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695450 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-binary-copy\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695459 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-multus-daemon-config\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-multus-cni-dir\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.695602 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695616 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-run\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.695677 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:44.195641248 +0000 UTC m=+3.012331031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695716 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-socket-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695740 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f2452907-e6ac-44a0-9f6b-d647c66dee6e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695771 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/27d45ca1-bef0-4f44-985b-679862b9856d-os-release\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695801 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-kubernetes\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.696688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-device-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695849 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-device-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695863 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-host\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6d5f375-8e25-4cad-8288-b127c8563011-ovn-node-metrics-cert\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.695969 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zfq8\" (UniqueName: \"kubernetes.io/projected/d0808476-c7a5-4036-b960-6ed233a8e75a-kube-api-access-6zfq8\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696066 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-cni-binary-copy\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696108 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-registration-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696157 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/782be745-9f10-411b-bb45-0f580202e2db-registration-dir\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-bin\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696766 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/27d45ca1-bef0-4f44-985b-679862b9856d-cni-binary-copy\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.697506 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.696827 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-tmp\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.698014 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.697763 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ff9cd805-64d9-4f13-9360-8461bb56d5b2-agent-certs\") pod \"konnectivity-agent-zhcbk\" (UID: \"ff9cd805-64d9-4f13-9360-8461bb56d5b2\") " pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.698229 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.698210 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-etc-tuned\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.701577 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.701535 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcth\" (UniqueName: \"kubernetes.io/projected/782be745-9f10-411b-bb45-0f580202e2db-kube-api-access-ftcth\") pod \"aws-ebs-csi-driver-node-dnhxc\" (UID: \"782be745-9f10-411b-bb45-0f580202e2db\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.701900 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.701883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tjwz\" (UniqueName: \"kubernetes.io/projected/75369ceb-252a-4d5e-b532-d20e91e04137-kube-api-access-5tjwz\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:43.701984 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.701898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtxh6\" (UniqueName: \"kubernetes.io/projected/f2452907-e6ac-44a0-9f6b-d647c66dee6e-kube-api-access-jtxh6\") pod \"multus-additional-cni-plugins-9hx4b\" (UID: \"f2452907-e6ac-44a0-9f6b-d647c66dee6e\") " pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.701984 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.701944 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl2hb\" (UniqueName: \"kubernetes.io/projected/e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0-kube-api-access-bl2hb\") pod \"tuned-gcvw4\" (UID: \"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0\") " pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.705205 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.705179 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwb74\" (UniqueName: \"kubernetes.io/projected/27d45ca1-bef0-4f44-985b-679862b9856d-kube-api-access-jwb74\") pod \"multus-qcdcv\" (UID: \"27d45ca1-bef0-4f44-985b-679862b9856d\") " pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.796933 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.796899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vslpb\" (UniqueName: \"kubernetes.io/projected/437089ae-aeec-4a9f-a4af-2c1237cb1357-kube-api-access-vslpb\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.797072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.796944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437089ae-aeec-4a9f-a4af-2c1237cb1357-host\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.797072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.796970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-config\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.796995 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-kubelet-config\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.797072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797026 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/437089ae-aeec-4a9f-a4af-2c1237cb1357-host\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.797072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-kubelet\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797080 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-slash\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797090 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-kubelet-config\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797133 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-slash\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6d5f375-8e25-4cad-8288-b127c8563011-ovn-node-metrics-cert\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797165 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-kubelet\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zfq8\" (UniqueName: \"kubernetes.io/projected/d0808476-c7a5-4036-b960-6ed233a8e75a-kube-api-access-6zfq8\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797172 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-bin\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.797295 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:43.797316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-netd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797327 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-bin\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.797345 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:44.297332186 +0000 UTC m=+3.114021970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437089ae-aeec-4a9f-a4af-2c1237cb1357-serviceca\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-iptables-alerter-script\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797374 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-cni-netd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797391 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-env-overrides\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-netns\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-log-socket\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797464 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797485 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-host-slash\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-systemd-units\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797516 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-run-netns\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drcmw\" (UniqueName: \"kubernetes.io/projected/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-kube-api-access-drcmw\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0808476-c7a5-4036-b960-6ed233a8e75a-tmp-dir\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.797897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-etc-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797599 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-config\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-var-lib-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-ovn\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797697 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-dbus\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-systemd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-node-log\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797784 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0808476-c7a5-4036-b960-6ed233a8e75a-hosts-file\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797810 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/437089ae-aeec-4a9f-a4af-2c1237cb1357-serviceca\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-script-lib\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797817 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-etc-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797595 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-log-socket\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797644 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-systemd-units\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7xw5\" (UniqueName: \"kubernetes.io/projected/f6d5f375-8e25-4cad-8288-b127c8563011-kube-api-access-k7xw5\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797874 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-ovn\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-run-systemd\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.798718 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797898 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0808476-c7a5-4036-b960-6ed233a8e75a-tmp-dir\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797918 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-var-lib-openvswitch\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797919 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-env-overrides\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797938 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-iptables-alerter-script\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797937 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797951 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-host-slash\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797982 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d0808476-c7a5-4036-b960-6ed233a8e75a-hosts-file\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.797994 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f6d5f375-8e25-4cad-8288-b127c8563011-node-log\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.798035 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a67614cc-102f-483b-858c-f180e0540da9-dbus\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:43.799520 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.798407 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f6d5f375-8e25-4cad-8288-b127c8563011-ovnkube-script-lib\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.799823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.799534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6d5f375-8e25-4cad-8288-b127c8563011-ovn-node-metrics-cert\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.804428 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.804407 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:43.804428 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.804429 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:43.804596 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.804443 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:43.804596 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:43.804498 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:44.304482071 +0000 UTC m=+3.121171867 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:43.806615 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.806590 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zfq8\" (UniqueName: \"kubernetes.io/projected/d0808476-c7a5-4036-b960-6ed233a8e75a-kube-api-access-6zfq8\") pod \"node-resolver-tlcwk\" (UID: \"d0808476-c7a5-4036-b960-6ed233a8e75a\") " pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.806805 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.806784 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drcmw\" (UniqueName: \"kubernetes.io/projected/a70f28f4-5ee0-4f15-a590-d5db13d9cec7-kube-api-access-drcmw\") pod \"iptables-alerter-jlt76\" (UID: \"a70f28f4-5ee0-4f15-a590-d5db13d9cec7\") " pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:43.807256 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.807239 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vslpb\" (UniqueName: \"kubernetes.io/projected/437089ae-aeec-4a9f-a4af-2c1237cb1357-kube-api-access-vslpb\") pod \"node-ca-jmkhh\" (UID: \"437089ae-aeec-4a9f-a4af-2c1237cb1357\") " pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.807322 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.807309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7xw5\" (UniqueName: \"kubernetes.io/projected/f6d5f375-8e25-4cad-8288-b127c8563011-kube-api-access-k7xw5\") pod \"ovnkube-node-78k77\" (UID: \"f6d5f375-8e25-4cad-8288-b127c8563011\") " pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.883523 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.883449 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" Jun 09 11:07:43.892759 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.892735 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qcdcv" Jun 09 11:07:43.900585 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.900556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" Jun 09 11:07:43.905225 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.905209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" Jun 09 11:07:43.910870 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.910849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:07:43.918644 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.918626 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:07:43.925202 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.925185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jmkhh" Jun 09 11:07:43.930696 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.930680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tlcwk" Jun 09 11:07:43.937162 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:43.937146 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jlt76" Jun 09 11:07:44.099854 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.099827 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 11:07:44.200324 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.200254 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:44.200471 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.200363 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:44.200471 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.200410 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:45.200397623 +0000 UTC m=+4.017087406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:44.301236 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.301209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:44.301393 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.301332 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:44.301393 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.301383 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:45.301367251 +0000 UTC m=+4.118057038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:44.401951 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.401873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:44.402021 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.401989 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:44.402021 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.402012 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:44.402100 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.402024 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:44.402100 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.402076 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:45.402062601 +0000 UTC m=+4.218752385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:44.416615 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.416556 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9cd805_64d9_4f13_9360_8461bb56d5b2.slice/crio-3c32b70e79e16634bce8342611530b3b1afa22477119946c8e5a7363d490bfa9 WatchSource:0}: Error finding container 3c32b70e79e16634bce8342611530b3b1afa22477119946c8e5a7363d490bfa9: Status 404 returned error can't find the container with id 3c32b70e79e16634bce8342611530b3b1afa22477119946c8e5a7363d490bfa9 Jun 09 11:07:44.417380 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.417358 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2452907_e6ac_44a0_9f6b_d647c66dee6e.slice/crio-aa8251bc82f5cf3d46af97cccf360f23532c63fe1e93c29e9cad45569f8d519d WatchSource:0}: Error finding container aa8251bc82f5cf3d46af97cccf360f23532c63fe1e93c29e9cad45569f8d519d: Status 404 returned error can't find the container with id aa8251bc82f5cf3d46af97cccf360f23532c63fe1e93c29e9cad45569f8d519d Jun 09 11:07:44.418879 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.418850 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d5f375_8e25_4cad_8288_b127c8563011.slice/crio-8b412579d8370786109ce789f5149c94f315c023edae2a9bba73826d71a0f40d WatchSource:0}: Error finding container 8b412579d8370786109ce789f5149c94f315c023edae2a9bba73826d71a0f40d: Status 404 returned error can't find the container with id 8b412579d8370786109ce789f5149c94f315c023edae2a9bba73826d71a0f40d Jun 09 11:07:44.421287 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.421262 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod437089ae_aeec_4a9f_a4af_2c1237cb1357.slice/crio-7a1c4097075df5a7678885c049a23af826e7763ed960b71faf2256be7183cdc1 WatchSource:0}: Error finding container 7a1c4097075df5a7678885c049a23af826e7763ed960b71faf2256be7183cdc1: Status 404 returned error can't find the container with id 7a1c4097075df5a7678885c049a23af826e7763ed960b71faf2256be7183cdc1 Jun 09 11:07:44.423677 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.423429 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f79525_22ac_4e8c_8d65_86b5d3f3fbe0.slice/crio-1eb955665eb09f8d375ae3ffb3e7603c2509248473b9302430a9fa9f83743615 WatchSource:0}: Error finding container 1eb955665eb09f8d375ae3ffb3e7603c2509248473b9302430a9fa9f83743615: Status 404 returned error can't find the container with id 1eb955665eb09f8d375ae3ffb3e7603c2509248473b9302430a9fa9f83743615 Jun 09 11:07:44.424187 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.424139 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0808476_c7a5_4036_b960_6ed233a8e75a.slice/crio-8a9d71735f99bd5fc28bd8f465990e0de934dd0a521179a7e41a69bfd95fe08d WatchSource:0}: Error finding container 8a9d71735f99bd5fc28bd8f465990e0de934dd0a521179a7e41a69bfd95fe08d: Status 404 returned error can't find the container with id 8a9d71735f99bd5fc28bd8f465990e0de934dd0a521179a7e41a69bfd95fe08d Jun 09 11:07:44.427370 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.427344 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d45ca1_bef0_4f44_985b_679862b9856d.slice/crio-b1da7e562368e02c66faff2bf99d1a7bdfcd6b6453aa9401ea230bd4167fdc51 WatchSource:0}: Error finding container b1da7e562368e02c66faff2bf99d1a7bdfcd6b6453aa9401ea230bd4167fdc51: Status 404 returned error can't find the container with id b1da7e562368e02c66faff2bf99d1a7bdfcd6b6453aa9401ea230bd4167fdc51 Jun 09 11:07:44.428001 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:07:44.427965 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782be745_9f10_411b_bb45_0f580202e2db.slice/crio-0f13c3338e8788a0daef51f7bea8be72cdd9d28d4c1a0aadcc52a1bb68051db4 WatchSource:0}: Error finding container 0f13c3338e8788a0daef51f7bea8be72cdd9d28d4c1a0aadcc52a1bb68051db4: Status 404 returned error can't find the container with id 0f13c3338e8788a0daef51f7bea8be72cdd9d28d4c1a0aadcc52a1bb68051db4 Jun 09 11:07:44.630648 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.630620 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 11:02:43 +0000 UTC" deadline="2028-01-19 18:33:28.572284112 +0000 UTC" Jun 09 11:07:44.630648 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.630648 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14143h25m43.941639207s" Jun 09 11:07:44.646140 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.646122 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:44.646280 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:44.646261 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:44.656173 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.656137 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" event={"ID":"79ba099ecd0a9bcdc399892bd5724baa","Type":"ContainerStarted","Data":"0b305d8790e1f791c0a7d87c1f381d78704a2e6f1ed4a442e02722ef9b749512"} Jun 09 11:07:44.657186 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.657164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlcwk" event={"ID":"d0808476-c7a5-4036-b960-6ed233a8e75a","Type":"ContainerStarted","Data":"8a9d71735f99bd5fc28bd8f465990e0de934dd0a521179a7e41a69bfd95fe08d"} Jun 09 11:07:44.658050 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.658032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" event={"ID":"782be745-9f10-411b-bb45-0f580202e2db","Type":"ContainerStarted","Data":"0f13c3338e8788a0daef51f7bea8be72cdd9d28d4c1a0aadcc52a1bb68051db4"} Jun 09 11:07:44.659081 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.659060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qcdcv" event={"ID":"27d45ca1-bef0-4f44-985b-679862b9856d","Type":"ContainerStarted","Data":"b1da7e562368e02c66faff2bf99d1a7bdfcd6b6453aa9401ea230bd4167fdc51"} Jun 09 11:07:44.660151 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.660130 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jlt76" event={"ID":"a70f28f4-5ee0-4f15-a590-d5db13d9cec7","Type":"ContainerStarted","Data":"a3948d73842143a8fb290ecb6c8c5c7970b6c249c4a6e9a4f277f401fedff10c"} Jun 09 11:07:44.661210 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.661193 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"8b412579d8370786109ce789f5149c94f315c023edae2a9bba73826d71a0f40d"} Jun 09 11:07:44.662122 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.662103 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zhcbk" event={"ID":"ff9cd805-64d9-4f13-9360-8461bb56d5b2","Type":"ContainerStarted","Data":"3c32b70e79e16634bce8342611530b3b1afa22477119946c8e5a7363d490bfa9"} Jun 09 11:07:44.663124 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.663108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" event={"ID":"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0","Type":"ContainerStarted","Data":"1eb955665eb09f8d375ae3ffb3e7603c2509248473b9302430a9fa9f83743615"} Jun 09 11:07:44.664100 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.664058 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmkhh" event={"ID":"437089ae-aeec-4a9f-a4af-2c1237cb1357","Type":"ContainerStarted","Data":"7a1c4097075df5a7678885c049a23af826e7763ed960b71faf2256be7183cdc1"} Jun 09 11:07:44.665085 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.665066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerStarted","Data":"aa8251bc82f5cf3d46af97cccf360f23532c63fe1e93c29e9cad45569f8d519d"} Jun 09 11:07:44.671389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:44.671349 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-8.ec2.internal" podStartSLOduration=2.6713383090000002 podStartE2EDuration="2.671338309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:07:44.671233293 +0000 UTC m=+3.487923098" watchObservedRunningTime="2026-06-09 11:07:44.671338309 +0000 UTC m=+3.488028113" Jun 09 11:07:45.208064 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.208024 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:45.208258 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.208196 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:45.208258 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.208256 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:47.20823898 +0000 UTC m=+6.024928777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:45.310593 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.308583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:45.310593 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.308748 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:45.310593 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.308808 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:47.308790859 +0000 UTC m=+6.125480656 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:45.408923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.408887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:45.409133 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.409115 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:45.409206 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.409142 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:45.409206 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.409156 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:45.409306 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.409225 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:47.409206965 +0000 UTC m=+6.225896762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:45.631232 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.631142 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 11:02:43 +0000 UTC" deadline="2028-03-06 06:09:08.805213752 +0000 UTC" Jun 09 11:07:45.631232 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.631229 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15259h1m23.173988525s" Jun 09 11:07:45.645809 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.645784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:45.645940 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.645907 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:45.646617 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.646598 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:45.646722 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:45.646703 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:45.677104 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.677076 2560 generic.go:356] "Generic (PLEG): container finished" podID="80a615a479690b22c773ee8338c8ef75" containerID="0eeb5aa2eafb4b95878f7b5abce42bf56fbc7284f2a3aae56c13ab23b9c41c87" exitCode=0 Jun 09 11:07:45.677975 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:45.677946 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"80a615a479690b22c773ee8338c8ef75","Type":"ContainerDied","Data":"0eeb5aa2eafb4b95878f7b5abce42bf56fbc7284f2a3aae56c13ab23b9c41c87"} Jun 09 11:07:46.646276 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:46.646232 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:46.646764 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:46.646365 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:46.704636 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:46.704600 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" event={"ID":"80a615a479690b22c773ee8338c8ef75","Type":"ContainerStarted","Data":"16a65db4100b1bcd1df750d01f1d9062ba2200ca5a8828f66f5b4aaca660b14b"} Jun 09 11:07:47.224645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:47.224606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:47.224817 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.224773 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:47.224877 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.224835 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:51.224817598 +0000 UTC m=+10.041507403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:47.325209 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:47.325169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:47.325378 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.325328 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:47.325441 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.325402 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:51.325382162 +0000 UTC m=+10.142071955 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:47.426638 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:47.426582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:47.426802 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.426709 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:47.426802 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.426735 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:47.426802 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.426749 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:47.426951 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.426818 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:51.426798108 +0000 UTC m=+10.243487913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:47.645728 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:47.645695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:47.645889 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.645824 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:47.646020 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:47.646001 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:47.646174 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:47.646094 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:48.645853 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:48.645812 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:48.646282 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:48.645952 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:49.645693 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:49.645662 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:49.645876 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:49.645784 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:49.646983 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:49.646958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:49.647155 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:49.647114 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:50.646188 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:50.646106 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:50.646688 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:50.646258 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:51.258659 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:51.258532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:51.258828 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.258709 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:51.258828 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.258783 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:59.258761956 +0000 UTC m=+18.075451751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:51.359580 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:51.359532 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:51.359763 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.359709 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:51.359837 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.359779 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:59.359760643 +0000 UTC m=+18.176450442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:51.461022 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:51.460849 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:51.461022 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.461015 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:51.461022 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.461034 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:51.461288 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.461046 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:51.461288 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.461101 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:07:59.461085414 +0000 UTC m=+18.277775197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:51.648289 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:51.648261 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:51.648870 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:51.648323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:51.649096 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.649060 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:51.649229 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:51.649123 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:52.646663 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:52.646292 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:52.646663 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:52.646441 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:53.646477 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:53.646440 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:53.646908 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:53.646490 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:53.646908 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:53.646601 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:53.646908 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:53.646664 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:54.646532 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:54.646506 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:54.646981 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:54.646647 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:55.646400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:55.646366 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:55.646662 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:55.646377 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:55.646662 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:55.646498 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:55.646662 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:55.646573 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:56.646204 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:56.646173 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:56.646366 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:56.646284 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:57.646249 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:57.646219 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:57.646700 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:57.646345 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:07:57.646700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:57.646384 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:57.646700 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:57.646484 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:58.646008 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:58.645977 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:58.646215 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:58.646101 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:07:59.318764 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:59.318727 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:07:59.319206 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.318878 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:59.319206 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.318948 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:15.318932967 +0000 UTC m=+34.135622753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:07:59.419799 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:59.419769 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:59.419970 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.419868 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:59.419970 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.419931 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:15.419914385 +0000 UTC m=+34.236604250 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:07:59.520893 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:59.520863 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:59.521059 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.520983 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:07:59.521059 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.521010 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:07:59.521059 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.521024 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:59.521169 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.521083 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:15.521068261 +0000 UTC m=+34.337758064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:07:59.646184 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:59.646102 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:07:59.646346 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.646232 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:07:59.646346 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:07:59.646265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:07:59.646462 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:07:59.646378 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:00.646411 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:00.646378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:00.646846 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:00.646501 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:01.646198 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:01.646165 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:01.646380 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:01.646260 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:01.646448 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:01.646385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:01.646755 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:01.646480 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:02.645783 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.645619 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:02.645888 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:02.645869 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:02.738608 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.738558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" event={"ID":"782be745-9f10-411b-bb45-0f580202e2db","Type":"ContainerStarted","Data":"8da59a817cfe4c0f2b522c7845faa433f23301d225c3a5284e20ccdb3c35ef92"} Jun 09 11:08:02.740068 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.740042 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qcdcv" event={"ID":"27d45ca1-bef0-4f44-985b-679862b9856d","Type":"ContainerStarted","Data":"f894ddee68c83d714b447d7c9838cc72ef3dd74f14e573d211cee4460f4d9a40"} Jun 09 11:08:02.741724 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.741706 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:08:02.742023 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.742004 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6d5f375-8e25-4cad-8288-b127c8563011" containerID="c66afd855269569277016d8be1a6906bc95cb0b6be86ca3facf0092bfa346693" exitCode=1 Jun 09 11:08:02.742103 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.742078 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"8174ec9ec21c0b4571f571082b257b081dadca3d83fa9d6ed8c4c633478f98f4"} Jun 09 11:08:02.742184 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.742162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerDied","Data":"c66afd855269569277016d8be1a6906bc95cb0b6be86ca3facf0092bfa346693"} Jun 09 11:08:02.742249 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.742228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"22467c9751df5c884264f61020cb35a7f92eb627441119203867fbe9b47d035c"} Jun 09 11:08:02.743347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.743326 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zhcbk" event={"ID":"ff9cd805-64d9-4f13-9360-8461bb56d5b2","Type":"ContainerStarted","Data":"fd7f8167f4f20ad063a6cd67d4fd00652e84ff1c50a8c3afb43af7c83cdd1d42"} Jun 09 11:08:02.744683 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.744662 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" event={"ID":"e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0","Type":"ContainerStarted","Data":"047f545092ec0d628ecb97ea1f6443b8369a9744e99c11ed70c867d4540f20ca"} Jun 09 11:08:02.745858 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.745841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jmkhh" event={"ID":"437089ae-aeec-4a9f-a4af-2c1237cb1357","Type":"ContainerStarted","Data":"91b34eacae43af5073aee4703f1039b41015209c9c6bb5418b91c2f420b5baca"} Jun 09 11:08:02.747269 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.747247 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="d40ee7fc593a38e3233c07edf5770ee45a9d415703cd710afc99c282be36abe8" exitCode=0 Jun 09 11:08:02.747357 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.747314 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"d40ee7fc593a38e3233c07edf5770ee45a9d415703cd710afc99c282be36abe8"} Jun 09 11:08:02.749067 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.749047 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tlcwk" event={"ID":"d0808476-c7a5-4036-b960-6ed233a8e75a","Type":"ContainerStarted","Data":"298beaca4792006240e0c011cd3fa036ea4a4296b39a1004ef1a1e13f418101c"} Jun 09 11:08:02.774323 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.774283 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-8.ec2.internal" podStartSLOduration=20.774270402 podStartE2EDuration="20.774270402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:07:46.726402456 +0000 UTC m=+5.543092261" watchObservedRunningTime="2026-06-09 11:08:02.774270402 +0000 UTC m=+21.590960206" Jun 09 11:08:02.791194 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.791161 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zhcbk" podStartSLOduration=12.375003012 podStartE2EDuration="21.791149497s" totalImagesPullingTime="9.416146485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:02.790703398 +0000 UTC m=+21.607393201" watchObservedRunningTime="2026-06-09 11:08:02.791149497 +0000 UTC m=+21.607839298" Jun 09 11:08:02.827462 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.827428 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jmkhh" podStartSLOduration=3.013616358 podStartE2EDuration="20.827417473s" totalImagesPullingTime="17.813801115s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:02.827180437 +0000 UTC m=+21.643870242" watchObservedRunningTime="2026-06-09 11:08:02.827417473 +0000 UTC m=+21.644107277" Jun 09 11:08:02.827708 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.827688 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gcvw4" podStartSLOduration=4.013690956 podStartE2EDuration="21.827680645s" totalImagesPullingTime="17.813989689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:02.80909435 +0000 UTC m=+21.625784156" watchObservedRunningTime="2026-06-09 11:08:02.827680645 +0000 UTC m=+21.644370452" Jun 09 11:08:02.870549 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:02.870516 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tlcwk" podStartSLOduration=3.057961249 podStartE2EDuration="20.870503614s" totalImagesPullingTime="17.812542365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:02.851473187 +0000 UTC m=+21.668162990" watchObservedRunningTime="2026-06-09 11:08:02.870503614 +0000 UTC m=+21.687193419" Jun 09 11:08:03.646353 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.646322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:03.646353 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.646354 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:03.646604 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:03.646438 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:03.646678 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:03.646594 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:03.754413 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.754212 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:08:03.755018 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.754991 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"3b68b30336231eaf896e6dc9d81512bacbf89dd8c3ba76664790fbccbe405f6a"} Jun 09 11:08:03.755114 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.755074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"220765a0bdcb44da16000b85f3067f3508b48223cc256f4da7235eb90de19ce0"} Jun 09 11:08:03.755179 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.755121 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"eef7acef78135841e736722a7061cb8af4d77113b92e3757d6599e6adb5901c9"} Jun 09 11:08:03.792274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:03.792253 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 11:08:04.586774 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.586745 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:08:04.587445 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.587427 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:08:04.607736 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.607686 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qcdcv" podStartSLOduration=5.778640822 podStartE2EDuration="23.607672958s" totalImagesPullingTime="17.829032136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:02.871321871 +0000 UTC m=+21.688011676" watchObservedRunningTime="2026-06-09 11:08:04.607672958 +0000 UTC m=+23.424362763" Jun 09 11:08:04.646548 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.646524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:04.646663 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.646493 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T11:08:03.792272074Z","UUID":"635bd02d-55fc-442b-a074-9089cc6f2a6e","Handler":null,"Name":"","Endpoint":""} Jun 09 11:08:04.646707 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:04.646654 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:04.647943 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.647924 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 11:08:04.648053 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.647952 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 11:08:04.758088 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.758063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" event={"ID":"782be745-9f10-411b-bb45-0f580202e2db","Type":"ContainerStarted","Data":"22afb2c1c4132d7d16be5763ceaf15fd267038badecbca83c40de8deefdf7ab4"} Jun 09 11:08:04.759536 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.759514 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jlt76" event={"ID":"a70f28f4-5ee0-4f15-a590-d5db13d9cec7","Type":"ContainerStarted","Data":"7bd88983b26755ca819fef3cb1157357e7a135bcfe7de87a5973c688a90eda39"} Jun 09 11:08:04.759747 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.759726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:08:04.760127 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.760111 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zhcbk" Jun 09 11:08:04.777469 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:04.777426 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jlt76" podStartSLOduration=4.966869345 podStartE2EDuration="22.77740903s" totalImagesPullingTime="17.810539685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:04.777281124 +0000 UTC m=+23.593970928" watchObservedRunningTime="2026-06-09 11:08:04.77740903 +0000 UTC m=+23.594098838" Jun 09 11:08:05.645915 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:05.645883 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:05.646111 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:05.645887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:05.646111 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:05.645999 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:05.646111 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:05.646071 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:05.764110 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:05.764074 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" event={"ID":"782be745-9f10-411b-bb45-0f580202e2db","Type":"ContainerStarted","Data":"964748e1ff36ad4c1fe155a4866c62a52f13a13aa005c5fba522a8a3cf12c19f"} Jun 09 11:08:05.767450 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:05.767425 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:08:05.767852 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:05.767827 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"b681e999b396df025d9c867040b888a4d98788678b54a2b6ef6300c4c09e11c4"} Jun 09 11:08:06.646416 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:06.646385 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:06.646593 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:06.646510 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:07.645692 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.645529 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:07.646184 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.645528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:07.646184 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:07.645770 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:07.646184 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:07.645827 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:07.773415 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.773390 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="fb9e15e2347c3faebcabc6354ce7fcf8a7b9740176610f0b49327ca64471aeab" exitCode=0 Jun 09 11:08:07.773542 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.773467 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"fb9e15e2347c3faebcabc6354ce7fcf8a7b9740176610f0b49327ca64471aeab"} Jun 09 11:08:07.776845 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.776831 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:08:07.777143 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.777118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"25d6e4a9d72a2b77a3587f14f46f558a4dbed9c14b90413a3ecc442913c0f325"} Jun 09 11:08:07.777363 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.777350 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:07.777409 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.777391 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:07.777495 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.777479 2560 scope.go:117] "RemoveContainer" containerID="c66afd855269569277016d8be1a6906bc95cb0b6be86ca3facf0092bfa346693" Jun 09 11:08:07.792471 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.792455 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:07.797604 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:07.797556 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dnhxc" podStartSLOduration=7.268941598 podStartE2EDuration="26.797543101s" totalImagesPullingTime="19.528601503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:05.786421642 +0000 UTC m=+24.603111450" watchObservedRunningTime="2026-06-09 11:08:07.797543101 +0000 UTC m=+26.614232904" Jun 09 11:08:08.646470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.646394 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:08.646815 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:08.646504 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:08.780795 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.780765 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="b1ccaedb57ee9989841775baf358896f57f87700ef97aa10527bc35c44cb3e81" exitCode=0 Jun 09 11:08:08.780937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.780848 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"b1ccaedb57ee9989841775baf358896f57f87700ef97aa10527bc35c44cb3e81"} Jun 09 11:08:08.784420 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.784401 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:08:08.784753 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.784735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" event={"ID":"f6d5f375-8e25-4cad-8288-b127c8563011","Type":"ContainerStarted","Data":"aa8cd7690f541a019eef161d4f76be321486ed02ce6adc2fde03efc09e9c2183"} Jun 09 11:08:08.784972 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.784952 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:08.799266 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.799248 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:08.831211 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:08.831144 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" podStartSLOduration=9.955274033 podStartE2EDuration="27.83113275s" totalImagesPullingTime="17.875858717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:08.829601207 +0000 UTC m=+27.646291012" watchObservedRunningTime="2026-06-09 11:08:08.83113275 +0000 UTC m=+27.647822534" Jun 09 11:08:09.645879 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:09.645859 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:09.646025 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:09.645940 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:09.646086 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:09.646024 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:09.646135 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:09.646095 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:09.790982 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:09.790955 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="c14d83e1bc45160af6a94191da767cbd8c8aa0bb96f615791a3a1ab77cec1a8e" exitCode=0 Jun 09 11:08:09.791273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:09.791041 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"c14d83e1bc45160af6a94191da767cbd8c8aa0bb96f615791a3a1ab77cec1a8e"} Jun 09 11:08:10.646241 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:10.646206 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:10.646398 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:10.646325 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:11.646865 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:11.646641 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:11.647314 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:11.646709 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:11.647314 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:11.646945 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:11.647314 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:11.647037 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:12.645713 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:12.645686 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:12.645873 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:12.645786 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:13.645959 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:13.645927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:13.646392 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:13.646069 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:13.646392 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:13.646110 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:13.646392 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:13.646231 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:14.646277 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:14.646244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:14.646657 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:14.646352 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:15.341274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.341247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:15.341391 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.341369 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:08:15.341434 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.341416 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs podName:75369ceb-252a-4d5e-b532-d20e91e04137 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:47.341403579 +0000 UTC m=+66.158093362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs") pod "network-metrics-daemon-t6fzv" (UID: "75369ceb-252a-4d5e-b532-d20e91e04137") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 11:08:15.383887 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.383855 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d2lzb"] Jun 09 11:08:15.384065 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.383956 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:15.384065 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.384052 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:15.385252 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.384987 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6fzv"] Jun 09 11:08:15.385356 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.385333 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:15.385464 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.385444 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:15.385728 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.385709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h928v"] Jun 09 11:08:15.385872 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.385787 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:15.385953 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.385871 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:15.441767 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.441713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:15.441860 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.441805 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 11:08:15.441860 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.441846 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret podName:a67614cc-102f-483b-858c-f180e0540da9 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:47.441833448 +0000 UTC m=+66.258523231 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret") pod "global-pull-secret-syncer-d2lzb" (UID: "a67614cc-102f-483b-858c-f180e0540da9") : object "kube-system"/"original-pull-secret" not registered Jun 09 11:08:15.542527 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.542503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:15.542655 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.542623 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 11:08:15.542655 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.542641 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 11:08:15.542655 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.542649 2560 projected.go:196] Error preparing data for projected volume kube-api-access-qv8pk for pod openshift-network-diagnostics/network-check-target-h928v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:08:15.542752 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:15.542691 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk podName:51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:47.542676017 +0000 UTC m=+66.359365817 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qv8pk" (UniqueName: "kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk") pod "network-check-target-h928v" (UID: "51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 11:08:15.806773 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.806718 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="e0673992832d7d33d3decfa214777e9450f7a75ec30fcff6da631ac02d998dfd" exitCode=0 Jun 09 11:08:15.807536 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:15.806801 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"e0673992832d7d33d3decfa214777e9450f7a75ec30fcff6da631ac02d998dfd"} Jun 09 11:08:16.645770 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:16.645557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:16.645997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:16.645557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:16.645997 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:16.645843 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:16.645997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:16.645557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:16.645997 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:16.645926 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:16.645997 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:16.645977 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:16.811690 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:16.811663 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2452907-e6ac-44a0-9f6b-d647c66dee6e" containerID="6ff6901e7deae18edca563766d049560a4874db18992aa22548d370826bb0be8" exitCode=0 Jun 09 11:08:16.811690 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:16.811693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerDied","Data":"6ff6901e7deae18edca563766d049560a4874db18992aa22548d370826bb0be8"} Jun 09 11:08:17.816242 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:17.816216 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" event={"ID":"f2452907-e6ac-44a0-9f6b-d647c66dee6e","Type":"ContainerStarted","Data":"1e4e4ea085ecf94c7225d4364cfad97aa4eeae18a28410ca07e1632bff825045"} Jun 09 11:08:17.846125 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:17.846065 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-9hx4b" podStartSLOduration=8.540810794 podStartE2EDuration="36.846052277s" totalImagesPullingTime="28.305241483s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:17.845677036 +0000 UTC m=+36.662366842" watchObservedRunningTime="2026-06-09 11:08:17.846052277 +0000 UTC m=+36.662742081" Jun 09 11:08:18.645658 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:18.645626 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:18.645658 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:18.645651 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:18.645870 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:18.645653 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:18.645870 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:18.645722 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h928v" podUID="51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5" Jun 09 11:08:18.645870 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:18.645794 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d2lzb" podUID="a67614cc-102f-483b-858c-f180e0540da9" Jun 09 11:08:18.645995 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:18.645901 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-t6fzv" podUID="75369ceb-252a-4d5e-b532-d20e91e04137" Jun 09 11:08:19.027433 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.027402 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-8.ec2.internal" event="NodeReady" Jun 09 11:08:19.027810 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.027507 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 11:08:19.093886 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.093843 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-d8sxj"] Jun 09 11:08:19.107661 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.107639 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gqdcp"] Jun 09 11:08:19.107776 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.107760 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.113078 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.113060 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 11:08:19.113180 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.113161 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qxgvb\"" Jun 09 11:08:19.113550 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.113535 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 11:08:19.113873 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.113856 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 11:08:19.119367 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.119349 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gqdcp"] Jun 09 11:08:19.119446 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.119377 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8sxj"] Jun 09 11:08:19.119446 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.119392 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4slfw"] Jun 09 11:08:19.119509 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.119452 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.122434 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.122416 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 11:08:19.122530 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.122452 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5k287\"" Jun 09 11:08:19.122530 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.122506 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 11:08:19.128164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.128148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.132364 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.132348 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 11:08:19.132471 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.132455 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 11:08:19.132867 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.132849 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 11:08:19.133302 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.133242 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 11:08:19.133662 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.133643 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-st677\"" Jun 09 11:08:19.134382 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.134365 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4slfw"] Jun 09 11:08:19.272267 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klv5r\" (UniqueName: \"kubernetes.io/projected/1b7df2e7-4a3c-4811-9251-26743ce760ac-kube-api-access-klv5r\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.272267 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.272267 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7df2e7-4a3c-4811-9251-26743ce760ac-tmp-dir\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.272486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-crio-socket\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.272486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7df2e7-4a3c-4811-9251-26743ce760ac-config-volume\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.272486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-cert\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.272486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272466 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b7df2e7-4a3c-4811-9251-26743ce760ac-metrics-tls\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.272646 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-data-volume\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.272646 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272505 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.272646 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwwxz\" (UniqueName: \"kubernetes.io/projected/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-api-access-kwwxz\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.272646 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.272545 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpgqk\" (UniqueName: \"kubernetes.io/projected/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-kube-api-access-wpgqk\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.373085 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klv5r\" (UniqueName: \"kubernetes.io/projected/1b7df2e7-4a3c-4811-9251-26743ce760ac-kube-api-access-klv5r\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.373215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7df2e7-4a3c-4811-9251-26743ce760ac-tmp-dir\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.373215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-crio-socket\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7df2e7-4a3c-4811-9251-26743ce760ac-config-volume\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-cert\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b7df2e7-4a3c-4811-9251-26743ce760ac-metrics-tls\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-data-volume\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373325 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwwxz\" (UniqueName: \"kubernetes.io/projected/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-api-access-kwwxz\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpgqk\" (UniqueName: \"kubernetes.io/projected/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-kube-api-access-wpgqk\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.373472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373390 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-crio-socket\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.373984 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.373957 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1b7df2e7-4a3c-4811-9251-26743ce760ac-tmp-dir\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.374092 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.374073 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-data-volume\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.374205 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.374131 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1b7df2e7-4a3c-4811-9251-26743ce760ac-config-volume\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.374325 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.374303 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.378953 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.378924 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.379047 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.378933 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1b7df2e7-4a3c-4811-9251-26743ce760ac-metrics-tls\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.379103 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.379047 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-cert\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.385654 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.385632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwwxz\" (UniqueName: \"kubernetes.io/projected/25a10ba9-83f5-4263-bb59-bf7bd3e16fd4-kube-api-access-kwwxz\") pod \"insights-runtime-extractor-4slfw\" (UID: \"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4\") " pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.385977 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.385961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klv5r\" (UniqueName: \"kubernetes.io/projected/1b7df2e7-4a3c-4811-9251-26743ce760ac-kube-api-access-klv5r\") pod \"dns-default-gqdcp\" (UID: \"1b7df2e7-4a3c-4811-9251-26743ce760ac\") " pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.386536 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.386521 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpgqk\" (UniqueName: \"kubernetes.io/projected/d014bbc8-b72e-48d9-bd8b-e6d6155bdccd-kube-api-access-wpgqk\") pod \"ingress-canary-d8sxj\" (UID: \"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd\") " pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.417292 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.417271 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-d8sxj" Jun 09 11:08:19.427457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.427436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:19.436619 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.436603 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4slfw" Jun 09 11:08:19.636843 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.636683 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-d8sxj"] Jun 09 11:08:19.646236 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:19.646195 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd014bbc8_b72e_48d9_bd8b_e6d6155bdccd.slice/crio-5b46188b0999703f21913b7aff747b317f4daacb7fd3fc52a17810a5ff5fee95 WatchSource:0}: Error finding container 5b46188b0999703f21913b7aff747b317f4daacb7fd3fc52a17810a5ff5fee95: Status 404 returned error can't find the container with id 5b46188b0999703f21913b7aff747b317f4daacb7fd3fc52a17810a5ff5fee95 Jun 09 11:08:19.649991 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.649820 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gqdcp"] Jun 09 11:08:19.653690 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:19.653668 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b7df2e7_4a3c_4811_9251_26743ce760ac.slice/crio-01839a51bade6fdbf671b9caa728bf985d8af689f0064738030a263ca50b04f8 WatchSource:0}: Error finding container 01839a51bade6fdbf671b9caa728bf985d8af689f0064738030a263ca50b04f8: Status 404 returned error can't find the container with id 01839a51bade6fdbf671b9caa728bf985d8af689f0064738030a263ca50b04f8 Jun 09 11:08:19.664982 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.664963 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4slfw"] Jun 09 11:08:19.667552 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:19.667531 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25a10ba9_83f5_4263_bb59_bf7bd3e16fd4.slice/crio-8c6e41d7a7f945364e0ff89063668b8062157216cbdfbd4f260182b615896980 WatchSource:0}: Error finding container 8c6e41d7a7f945364e0ff89063668b8062157216cbdfbd4f260182b615896980: Status 404 returned error can't find the container with id 8c6e41d7a7f945364e0ff89063668b8062157216cbdfbd4f260182b615896980 Jun 09 11:08:19.822273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.822233 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8sxj" event={"ID":"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd","Type":"ContainerStarted","Data":"5b46188b0999703f21913b7aff747b317f4daacb7fd3fc52a17810a5ff5fee95"} Jun 09 11:08:19.823668 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.823644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4slfw" event={"ID":"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4","Type":"ContainerStarted","Data":"9e9829e91d1099e6a06a7288315908c802095cc05a9568d2f44d64bfc1292699"} Jun 09 11:08:19.823751 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.823696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4slfw" event={"ID":"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4","Type":"ContainerStarted","Data":"8c6e41d7a7f945364e0ff89063668b8062157216cbdfbd4f260182b615896980"} Jun 09 11:08:19.824653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:19.824627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqdcp" event={"ID":"1b7df2e7-4a3c-4811-9251-26743ce760ac","Type":"ContainerStarted","Data":"01839a51bade6fdbf671b9caa728bf985d8af689f0064738030a263ca50b04f8"} Jun 09 11:08:20.646014 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.645925 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:20.646014 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.645976 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:20.647001 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.646042 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:20.651043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.650608 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 11:08:20.651043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.650628 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-464pd\"" Jun 09 11:08:20.651043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.650869 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ft8nx\"" Jun 09 11:08:20.651305 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.651058 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 11:08:20.651305 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.651246 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 11:08:20.651643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:20.651455 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 11:08:21.738329 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.738298 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9249x"] Jun 09 11:08:21.749220 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.749199 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.751636 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.751512 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9249x"] Jun 09 11:08:21.752046 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.751995 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 11:08:21.752046 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.752004 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 09 11:08:21.752046 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.752010 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 11:08:21.752235 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.752069 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 09 11:08:21.752235 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.752080 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 11:08:21.753238 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.753221 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-crzgk\"" Jun 09 11:08:21.890062 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.890037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.890135 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.890067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d143709-9689-4915-a318-709894827de4-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.890135 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.890091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swmrn\" (UniqueName: \"kubernetes.io/projected/3d143709-9689-4915-a318-709894827de4-kube-api-access-swmrn\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.890198 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.890133 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.990957 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.990937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.991036 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.990965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d143709-9689-4915-a318-709894827de4-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.991036 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.990988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swmrn\" (UniqueName: \"kubernetes.io/projected/3d143709-9689-4915-a318-709894827de4-kube-api-access-swmrn\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.991036 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.991030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.991156 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:21.991143 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 09 11:08:21.991207 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:21.991198 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls podName:3d143709-9689-4915-a318-709894827de4 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:22.491182111 +0000 UTC m=+41.307871895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-9249x" (UID: "3d143709-9689-4915-a318-709894827de4") : secret "prometheus-operator-tls" not found Jun 09 11:08:21.991494 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.991470 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d143709-9689-4915-a318-709894827de4-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:21.993966 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:21.993936 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:22.004493 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.004468 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swmrn\" (UniqueName: \"kubernetes.io/projected/3d143709-9689-4915-a318-709894827de4-kube-api-access-swmrn\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:22.495481 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.495451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:22.497627 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.497609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/3d143709-9689-4915-a318-709894827de4-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-9249x\" (UID: \"3d143709-9689-4915-a318-709894827de4\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:22.659475 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.659448 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" Jun 09 11:08:22.791931 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.791898 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-9249x"] Jun 09 11:08:22.794669 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:22.794644 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d143709_9689_4915_a318_709894827de4.slice/crio-8dee3e76f599d53a2f2340e1cd2371059f70fe38f89c418bd92b4a00e1740753 WatchSource:0}: Error finding container 8dee3e76f599d53a2f2340e1cd2371059f70fe38f89c418bd92b4a00e1740753: Status 404 returned error can't find the container with id 8dee3e76f599d53a2f2340e1cd2371059f70fe38f89c418bd92b4a00e1740753 Jun 09 11:08:22.834712 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.834676 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-d8sxj" event={"ID":"d014bbc8-b72e-48d9-bd8b-e6d6155bdccd","Type":"ContainerStarted","Data":"8b0db3b0b1f09e638b8bfa23222553262fb54c18b518aa66087bb63bec27dc4f"} Jun 09 11:08:22.835805 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.835776 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" event={"ID":"3d143709-9689-4915-a318-709894827de4","Type":"ContainerStarted","Data":"8dee3e76f599d53a2f2340e1cd2371059f70fe38f89c418bd92b4a00e1740753"} Jun 09 11:08:22.837319 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.837300 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4slfw" event={"ID":"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4","Type":"ContainerStarted","Data":"22e12d96e5b86924ea14280e843482643c0dc1662a4b5549dea2504e828aa8a9"} Jun 09 11:08:22.838859 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.838837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqdcp" event={"ID":"1b7df2e7-4a3c-4811-9251-26743ce760ac","Type":"ContainerStarted","Data":"dd16ad16a0de7dd437df8500f0591a06990d818608eb4a84bd009a29e53d8f3d"} Jun 09 11:08:22.838958 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.838878 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gqdcp" event={"ID":"1b7df2e7-4a3c-4811-9251-26743ce760ac","Type":"ContainerStarted","Data":"4beba98c0573cd7b4e7200c1c9878714eebae7dc69158cab762fd015fe86480b"} Jun 09 11:08:22.839037 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.838980 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:22.852252 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.852215 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-d8sxj" podStartSLOduration=1.51228943 podStartE2EDuration="3.852203873s" totalImagesPullingTime="2.339914443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:22.851883062 +0000 UTC m=+41.668572866" watchObservedRunningTime="2026-06-09 11:08:22.852203873 +0000 UTC m=+41.668893676" Jun 09 11:08:22.872311 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:22.872270 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gqdcp" podStartSLOduration=1.545001882 podStartE2EDuration="3.872259902s" totalImagesPullingTime="2.32725802s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:22.872056025 +0000 UTC m=+41.688745849" watchObservedRunningTime="2026-06-09 11:08:22.872259902 +0000 UTC m=+41.688949686" Jun 09 11:08:25.849784 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:25.849747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" event={"ID":"3d143709-9689-4915-a318-709894827de4","Type":"ContainerStarted","Data":"a744c7f3383fc5f9bda3f8237bdaac8da831339167a1095852e8b9bfa9b1f4c3"} Jun 09 11:08:25.850261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:25.849803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" event={"ID":"3d143709-9689-4915-a318-709894827de4","Type":"ContainerStarted","Data":"a9bc3689ed7ca1ab04137dfb83620147a52184827cc56e2cfa2b16a41c5f4bc5"} Jun 09 11:08:25.851703 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:25.851680 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4slfw" event={"ID":"25a10ba9-83f5-4263-bb59-bf7bd3e16fd4","Type":"ContainerStarted","Data":"37842457154b40d147b93ae5fe584dd73924c8da214006e8811b224c296220bc"} Jun 09 11:08:25.867685 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:25.867638 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-9249x" podStartSLOduration=2.534308454 podStartE2EDuration="4.867624246s" totalImagesPullingTime="2.333315792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:25.866709372 +0000 UTC m=+44.683399176" watchObservedRunningTime="2026-06-09 11:08:25.867624246 +0000 UTC m=+44.684314171" Jun 09 11:08:25.885376 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:25.885341 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4slfw" podStartSLOduration=1.657055214 podStartE2EDuration="6.885329236s" totalImagesPullingTime="5.228274022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:08:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:25.885237121 +0000 UTC m=+44.701926926" watchObservedRunningTime="2026-06-09 11:08:25.885329236 +0000 UTC m=+44.702019039" Jun 09 11:08:28.255819 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.255787 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8g78d"] Jun 09 11:08:28.284201 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.284174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.287751 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.287732 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 11:08:28.287751 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.287746 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z8srh\"" Jun 09 11:08:28.287900 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.287746 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 11:08:28.287900 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.287806 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 11:08:28.332421 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-root\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332447 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-sys\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332496 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332523 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gpkp\" (UniqueName: \"kubernetes.io/projected/f376332f-bbb7-4e06-97a1-a7a0f10674a7-kube-api-access-9gpkp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-metrics-client-ca\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-textfile\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.332706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.332665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-wtmp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433615 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433580 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433615 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gpkp\" (UniqueName: \"kubernetes.io/projected/f376332f-bbb7-4e06-97a1-a7a0f10674a7-kube-api-access-9gpkp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433813 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433869 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-metrics-client-ca\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433869 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433859 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-textfile\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433969 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-wtmp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.433969 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-root\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434069 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433974 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-root\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434069 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.433974 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434069 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-sys\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434069 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:28.434051 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 09 11:08:28.434069 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-wtmp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434294 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434083 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f376332f-bbb7-4e06-97a1-a7a0f10674a7-sys\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434294 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:28.434109 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls podName:f376332f-bbb7-4e06-97a1-a7a0f10674a7 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:28.934089709 +0000 UTC m=+47.750779496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls") pod "node-exporter-8g78d" (UID: "f376332f-bbb7-4e06-97a1-a7a0f10674a7") : secret "node-exporter-tls" not found Jun 09 11:08:28.434294 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434236 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-textfile\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434443 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434362 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-accelerators-collector-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.434443 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.434383 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f376332f-bbb7-4e06-97a1-a7a0f10674a7-metrics-client-ca\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.435930 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.435910 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.442173 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.442150 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gpkp\" (UniqueName: \"kubernetes.io/projected/f376332f-bbb7-4e06-97a1-a7a0f10674a7-kube-api-access-9gpkp\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.937328 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.937290 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:28.939350 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:28.939326 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f376332f-bbb7-4e06-97a1-a7a0f10674a7-node-exporter-tls\") pod \"node-exporter-8g78d\" (UID: \"f376332f-bbb7-4e06-97a1-a7a0f10674a7\") " pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:29.193145 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.193078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8g78d" Jun 09 11:08:29.210059 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.210028 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 11:08:29.226643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.226615 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.227325 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.227302 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 11:08:29.228843 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:29.228818 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf376332f_bbb7_4e06_97a1_a7a0f10674a7.slice/crio-141fe65dfe4957cde01cd2b1217da33a4f5e270fcee5df62537c0ec48aa98712 WatchSource:0}: Error finding container 141fe65dfe4957cde01cd2b1217da33a4f5e270fcee5df62537c0ec48aa98712: Status 404 returned error can't find the container with id 141fe65dfe4957cde01cd2b1217da33a4f5e270fcee5df62537c0ec48aa98712 Jun 09 11:08:29.231262 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.230974 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 09 11:08:29.231262 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231029 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 09 11:08:29.231262 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231048 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 09 11:08:29.231262 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231123 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 09 11:08:29.231531 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231362 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q9cp4\"" Jun 09 11:08:29.231679 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231547 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 09 11:08:29.231931 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231766 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 09 11:08:29.231931 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231838 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 09 11:08:29.231931 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231863 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 09 11:08:29.232140 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.231972 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 09 11:08:29.239075 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239057 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239183 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239169 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-647th\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-kube-api-access-647th\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239363 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-out\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239601 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239601 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239516 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239601 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239601 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239791 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239611 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-web-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239791 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.239791 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.239686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.340920 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.340888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.340928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-647th\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-kube-api-access-647th\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.340958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.340983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-out\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341036 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:29.341157 2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:29.341216 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls podName:2b57d2dc-6df7-416a-96e0-59476b49d6c9 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:29.841197541 +0000 UTC m=+48.657887337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "2b57d2dc-6df7-416a-96e0-59476b49d6c9") : secret "alertmanager-main-tls" not found Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341279 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-web-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341335 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.341710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.341892 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:08:29.341827 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle podName:2b57d2dc-6df7-416a-96e0-59476b49d6c9 nodeName:}" failed. No retries permitted until 2026-06-09 11:08:29.841811055 +0000 UTC m=+48.658500842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "2b57d2dc-6df7-416a-96e0-59476b49d6c9") : configmap references non-existent config key: ca-bundle.crt Jun 09 11:08:29.342167 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.342057 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344006 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344005 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344222 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344191 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-out\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344459 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344438 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344521 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344586 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-web-config\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344586 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.344969 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.344954 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.349812 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.349792 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-647th\" (UniqueName: \"kubernetes.io/projected/2b57d2dc-6df7-416a-96e0-59476b49d6c9-kube-api-access-647th\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.845907 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.845867 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.846104 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.845928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.846686 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.846659 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b57d2dc-6df7-416a-96e0-59476b49d6c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.848471 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.848448 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2b57d2dc-6df7-416a-96e0-59476b49d6c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2b57d2dc-6df7-416a-96e0-59476b49d6c9\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:29.868484 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:29.868457 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8g78d" event={"ID":"f376332f-bbb7-4e06-97a1-a7a0f10674a7","Type":"ContainerStarted","Data":"141fe65dfe4957cde01cd2b1217da33a4f5e270fcee5df62537c0ec48aa98712"} Jun 09 11:08:30.144915 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:30.144842 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 11:08:30.365913 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:30.365888 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 11:08:30.367415 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:30.367384 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b57d2dc_6df7_416a_96e0_59476b49d6c9.slice/crio-d382acb3a5878550250c31fbc1c4ee84344c864537af7f293b17eac5d657eef4 WatchSource:0}: Error finding container d382acb3a5878550250c31fbc1c4ee84344c864537af7f293b17eac5d657eef4: Status 404 returned error can't find the container with id d382acb3a5878550250c31fbc1c4ee84344c864537af7f293b17eac5d657eef4 Jun 09 11:08:30.874998 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:30.874919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"d382acb3a5878550250c31fbc1c4ee84344c864537af7f293b17eac5d657eef4"} Jun 09 11:08:30.876721 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:30.876692 2560 generic.go:356] "Generic (PLEG): container finished" podID="f376332f-bbb7-4e06-97a1-a7a0f10674a7" containerID="4eea448274c5fe3167d6ff9b19264e306837e5d5bdfe95722ecdc181e45ec44f" exitCode=0 Jun 09 11:08:30.876846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:30.876741 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8g78d" event={"ID":"f376332f-bbb7-4e06-97a1-a7a0f10674a7","Type":"ContainerDied","Data":"4eea448274c5fe3167d6ff9b19264e306837e5d5bdfe95722ecdc181e45ec44f"} Jun 09 11:08:31.137578 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.137474 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-56b7cb4894-kzg88"] Jun 09 11:08:31.153181 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.153155 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.154017 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.153994 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56b7cb4894-kzg88"] Jun 09 11:08:31.156391 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156204 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 09 11:08:31.156391 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156265 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 09 11:08:31.156391 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156205 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 09 11:08:31.156996 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156667 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-7uco4lk577skt\"" Jun 09 11:08:31.156996 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156699 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 09 11:08:31.156996 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.156944 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-75k5z\"" Jun 09 11:08:31.157148 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.157068 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 09 11:08:31.256744 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.256880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/641bc884-e723-49c9-82c5-79b27c994999-metrics-client-ca\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.256880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256781 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.256880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.257024 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.257024 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.257024 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.256960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4c5r\" (UniqueName: \"kubernetes.io/projected/641bc884-e723-49c9-82c5-79b27c994999-kube-api-access-j4c5r\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.257133 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.257043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-grpc-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358052 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358018 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358052 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358055 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/641bc884-e723-49c9-82c5-79b27c994999-metrics-client-ca\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358077 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358160 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4c5r\" (UniqueName: \"kubernetes.io/projected/641bc884-e723-49c9-82c5-79b27c994999-kube-api-access-j4c5r\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358211 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-grpc-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.358846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.358795 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/641bc884-e723-49c9-82c5-79b27c994999-metrics-client-ca\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.360659 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.360480 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.360775 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.360675 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.360845 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.360827 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.361025 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.361001 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.361229 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.361210 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-thanos-querier-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.361385 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.361367 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/641bc884-e723-49c9-82c5-79b27c994999-secret-grpc-tls\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.366733 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.366716 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4c5r\" (UniqueName: \"kubernetes.io/projected/641bc884-e723-49c9-82c5-79b27c994999-kube-api-access-j4c5r\") pod \"thanos-querier-56b7cb4894-kzg88\" (UID: \"641bc884-e723-49c9-82c5-79b27c994999\") " pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.464028 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.463971 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:31.597223 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.597201 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-56b7cb4894-kzg88"] Jun 09 11:08:31.599469 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:31.599433 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod641bc884_e723_49c9_82c5_79b27c994999.slice/crio-87e40dbddc7a4fef73d65af718e18b9c39220e0ecf97225cb84a1b0559ab1ea6 WatchSource:0}: Error finding container 87e40dbddc7a4fef73d65af718e18b9c39220e0ecf97225cb84a1b0559ab1ea6: Status 404 returned error can't find the container with id 87e40dbddc7a4fef73d65af718e18b9c39220e0ecf97225cb84a1b0559ab1ea6 Jun 09 11:08:31.881609 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.881585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8g78d" event={"ID":"f376332f-bbb7-4e06-97a1-a7a0f10674a7","Type":"ContainerStarted","Data":"01282161c50912db867ee842c3b2a7f617febbedf1b6e6ea8bcb8ecb1ab136ef"} Jun 09 11:08:31.881717 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.881631 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8g78d" event={"ID":"f376332f-bbb7-4e06-97a1-a7a0f10674a7","Type":"ContainerStarted","Data":"ce991cfaa9876770aabb0c1fb8cbea8408a4b99331edaee0695dd4127dc4a99d"} Jun 09 11:08:31.882748 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.882725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"87e40dbddc7a4fef73d65af718e18b9c39220e0ecf97225cb84a1b0559ab1ea6"} Jun 09 11:08:31.916872 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:31.916834 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8g78d" podStartSLOduration=2.4686447449999998 podStartE2EDuration="3.916819309s" totalImagesPullingTime="1.346504344s" totalInitContainerRuntime="101.67022ms" isStatefulPod=true podCreationTimestamp="2026-06-09 11:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:31.916451772 +0000 UTC m=+50.733141576" watchObservedRunningTime="2026-06-09 11:08:31.916819309 +0000 UTC m=+50.733509115" Jun 09 11:08:32.467579 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.467538 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6ddfbcd75b-hch67"] Jun 09 11:08:32.481762 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.481692 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ddfbcd75b-hch67"] Jun 09 11:08:32.481889 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.481828 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486421 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486434 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-f6ntz\"" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486470 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6fq783h66pgqd\"" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486491 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486422 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 09 11:08:32.486645 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.486628 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 11:08:32.569700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569675 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-tls\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.569846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/213c7c8e-26bc-4038-be15-ed1bcc4480ae-audit-log\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.569905 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-client-certs\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.569905 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-metrics-server-audit-profiles\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.570010 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569927 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.570010 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.569994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-client-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.570111 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.570027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfgp4\" (UniqueName: \"kubernetes.io/projected/213c7c8e-26bc-4038-be15-ed1bcc4480ae-kube-api-access-jfgp4\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-client-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670763 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfgp4\" (UniqueName: \"kubernetes.io/projected/213c7c8e-26bc-4038-be15-ed1bcc4480ae-kube-api-access-jfgp4\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670763 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670671 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-tls\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670763 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/213c7c8e-26bc-4038-be15-ed1bcc4480ae-audit-log\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670913 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-client-certs\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670913 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-metrics-server-audit-profiles\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.670913 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.670856 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.671217 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.671188 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/213c7c8e-26bc-4038-be15-ed1bcc4480ae-audit-log\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.671550 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.671527 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.672003 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.671981 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/213c7c8e-26bc-4038-be15-ed1bcc4480ae-metrics-server-audit-profiles\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.673389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.673365 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-client-ca-bundle\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.673577 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.673545 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-tls\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.673637 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.673616 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/213c7c8e-26bc-4038-be15-ed1bcc4480ae-secret-metrics-server-client-certs\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.689593 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.689556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfgp4\" (UniqueName: \"kubernetes.io/projected/213c7c8e-26bc-4038-be15-ed1bcc4480ae-kube-api-access-jfgp4\") pod \"metrics-server-6ddfbcd75b-hch67\" (UID: \"213c7c8e-26bc-4038-be15-ed1bcc4480ae\") " pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.793724 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.793690 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:32.845825 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.845800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gqdcp" Jun 09 11:08:32.891331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.891297 2560 generic.go:356] "Generic (PLEG): container finished" podID="2b57d2dc-6df7-416a-96e0-59476b49d6c9" containerID="efd30a84487ef73732844cf67e850c453a68811276201293aff60ce0e385ff8c" exitCode=0 Jun 09 11:08:32.891462 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.891388 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerDied","Data":"efd30a84487ef73732844cf67e850c453a68811276201293aff60ce0e385ff8c"} Jun 09 11:08:32.967552 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.967476 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6ddfbcd75b-hch67"] Jun 09 11:08:32.969375 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:32.969353 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213c7c8e_26bc_4038_be15_ed1bcc4480ae.slice/crio-bd871dba1a89e747c8f4111015a42d316d060ca9db6df575cc8fbb3e140acd87 WatchSource:0}: Error finding container bd871dba1a89e747c8f4111015a42d316d060ca9db6df575cc8fbb3e140acd87: Status 404 returned error can't find the container with id bd871dba1a89e747c8f4111015a42d316d060ca9db6df575cc8fbb3e140acd87 Jun 09 11:08:32.984995 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:32.984934 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7"] Jun 09 11:08:33.015384 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.015362 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7"] Jun 09 11:08:33.015470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.015444 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.017901 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.017874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9krb2\"" Jun 09 11:08:33.018273 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.018259 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 09 11:08:33.075195 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.075134 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6fdd4b06-f90f-4858-8716-534477fe901d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.075195 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.075160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5hbk\" (UniqueName: \"kubernetes.io/projected/6fdd4b06-f90f-4858-8716-534477fe901d-kube-api-access-n5hbk\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.176122 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.176095 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6fdd4b06-f90f-4858-8716-534477fe901d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.176257 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.176125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5hbk\" (UniqueName: \"kubernetes.io/projected/6fdd4b06-f90f-4858-8716-534477fe901d-kube-api-access-n5hbk\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.178287 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.178269 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6fdd4b06-f90f-4858-8716-534477fe901d-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.185452 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.185429 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5hbk\" (UniqueName: \"kubernetes.io/projected/6fdd4b06-f90f-4858-8716-534477fe901d-kube-api-access-n5hbk\") pod \"monitoring-plugin-76b68cc874-sxrd7\" (UID: \"6fdd4b06-f90f-4858-8716-534477fe901d\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.324469 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.324436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:33.456104 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.456069 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-dd644d585-7gn4g"] Jun 09 11:08:33.471196 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.471161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.471637 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.471264 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dd644d585-7gn4g"] Jun 09 11:08:33.474058 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.473951 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 09 11:08:33.474058 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.473973 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 09 11:08:33.474242 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.474226 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 09 11:08:33.474307 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.474277 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 09 11:08:33.475309 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.475166 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g4w4h\"" Jun 09 11:08:33.475309 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.475203 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 09 11:08:33.479417 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.479395 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 09 11:08:33.580004 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.579970 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwnr8\" (UniqueName: \"kubernetes.io/projected/e266599b-4e54-4441-9ee8-50dfbf229f56-kube-api-access-zwnr8\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580159 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-metrics-client-ca\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580159 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580047 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580159 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580330 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580330 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580250 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580330 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-serving-certs-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.580330 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.580326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-federate-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681252 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-metrics-client-ca\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681252 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681476 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681297 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681476 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681476 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681476 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681442 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-serving-certs-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-federate-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.681706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.681527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwnr8\" (UniqueName: \"kubernetes.io/projected/e266599b-4e54-4441-9ee8-50dfbf229f56-kube-api-access-zwnr8\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.682884 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.682836 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-metrics-client-ca\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.683319 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.683287 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-trusted-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.683781 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.683739 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e266599b-4e54-4441-9ee8-50dfbf229f56-serving-certs-ca-bundle\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.685557 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.685491 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.685908 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.685886 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-secret-telemeter-client\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.686411 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.686362 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-federate-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.687694 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.687672 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e266599b-4e54-4441-9ee8-50dfbf229f56-telemeter-client-tls\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.690906 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.690884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwnr8\" (UniqueName: \"kubernetes.io/projected/e266599b-4e54-4441-9ee8-50dfbf229f56-kube-api-access-zwnr8\") pod \"telemeter-client-dd644d585-7gn4g\" (UID: \"e266599b-4e54-4441-9ee8-50dfbf229f56\") " pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.782434 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.782404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" Jun 09 11:08:33.900929 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.900889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" event={"ID":"213c7c8e-26bc-4038-be15-ed1bcc4480ae","Type":"ContainerStarted","Data":"bd871dba1a89e747c8f4111015a42d316d060ca9db6df575cc8fbb3e140acd87"} Jun 09 11:08:33.981047 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.981015 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-dd644d585-7gn4g"] Jun 09 11:08:33.986376 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:33.986321 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode266599b_4e54_4441_9ee8_50dfbf229f56.slice/crio-1f1e7827c65701dfc9087c752204dadc5ae93b0b7a2be5b667b9ae514a4beab6 WatchSource:0}: Error finding container 1f1e7827c65701dfc9087c752204dadc5ae93b0b7a2be5b667b9ae514a4beab6: Status 404 returned error can't find the container with id 1f1e7827c65701dfc9087c752204dadc5ae93b0b7a2be5b667b9ae514a4beab6 Jun 09 11:08:33.999503 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:33.999395 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7"] Jun 09 11:08:34.001794 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:34.001767 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdd4b06_f90f_4858_8716_534477fe901d.slice/crio-48d7f70f4c454bd0cdb926f5972dae7c45f435befbddfb9ff1ee08c07cf629fe WatchSource:0}: Error finding container 48d7f70f4c454bd0cdb926f5972dae7c45f435befbddfb9ff1ee08c07cf629fe: Status 404 returned error can't find the container with id 48d7f70f4c454bd0cdb926f5972dae7c45f435befbddfb9ff1ee08c07cf629fe Jun 09 11:08:34.339960 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.339610 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 11:08:34.357921 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.357860 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.361303 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.360970 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 11:08:34.361303 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.360995 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 11:08:34.361303 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.361153 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 11:08:34.361303 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.361288 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 11:08:34.362206 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.362144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-debeqnp1508kn\"" Jun 09 11:08:34.362413 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.362291 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 11:08:34.362413 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.362332 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 11:08:34.362849 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.362477 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 11:08:34.362849 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.362518 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.364744 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.364882 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.365087 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.365104 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.365322 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.365357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4wrkv\"" Jun 09 11:08:34.366215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.365536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 11:08:34.367265 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.367246 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 11:08:34.387855 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.387830 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388124 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388329 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388420 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388354 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388420 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388381 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388420 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388408 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388603 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388603 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388603 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.388603 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388752 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388932 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.389150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.388955 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc7d4\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-kube-api-access-hc7d4\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490081 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490127 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490187 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490487 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490541 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490683 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490706 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc7d4\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-kube-api-access-hc7d4\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490740 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.490970 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.490957 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.498869 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.498811 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.499430 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.499405 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.499785 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.499851 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.499952 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500007 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500283 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500117 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500339 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500302 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500459 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500416 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-web-config\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500715 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500698 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500862 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500820 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.500943 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500901 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.501008 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.500963 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.501464 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.501360 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.501464 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.501388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.501934 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.501892 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-config-out\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.502099 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.502066 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.502987 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.502928 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc7d4\" (UniqueName: \"kubernetes.io/projected/ef6e1b10-50c4-45d2-b53a-918ba01e65e7-kube-api-access-hc7d4\") pod \"prometheus-k8s-0\" (UID: \"ef6e1b10-50c4-45d2-b53a-918ba01e65e7\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.673708 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.673635 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:34.884595 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.884546 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 11:08:34.905418 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.905392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" event={"ID":"e266599b-4e54-4441-9ee8-50dfbf229f56","Type":"ContainerStarted","Data":"1f1e7827c65701dfc9087c752204dadc5ae93b0b7a2be5b667b9ae514a4beab6"} Jun 09 11:08:34.906818 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.906794 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"df524ecdb003c5c1934e22e44ce3b6c26c3ea9153fb74dd43ea2372cbf7602f9"} Jun 09 11:08:34.907754 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:34.907736 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" event={"ID":"6fdd4b06-f90f-4858-8716-534477fe901d","Type":"ContainerStarted","Data":"48d7f70f4c454bd0cdb926f5972dae7c45f435befbddfb9ff1ee08c07cf629fe"} Jun 09 11:08:35.025002 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:35.024934 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6e1b10_50c4_45d2_b53a_918ba01e65e7.slice/crio-0bc7ea2f8d216b49709892cb41e59bf618a7d1b6d7a269bca899f1a15e4c48ae WatchSource:0}: Error finding container 0bc7ea2f8d216b49709892cb41e59bf618a7d1b6d7a269bca899f1a15e4c48ae: Status 404 returned error can't find the container with id 0bc7ea2f8d216b49709892cb41e59bf618a7d1b6d7a269bca899f1a15e4c48ae Jun 09 11:08:35.914142 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:35.914073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"554a6d1f70ae8ac97e3a685be980ddd06e9e091fabce2cda649f62f5ba3c6f6f"} Jun 09 11:08:35.915834 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:35.915806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"8777eee24fe6109b5e5627be8574f9d2c86e0271a9d9d6865afa4b5b2959ddf0"} Jun 09 11:08:35.917214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:35.917192 2560 generic.go:356] "Generic (PLEG): container finished" podID="ef6e1b10-50c4-45d2-b53a-918ba01e65e7" containerID="c88e16acdada7ec1c339855e105e88c0cc9ff1de38d437d57442e12c499e26fd" exitCode=0 Jun 09 11:08:35.917338 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:35.917226 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerDied","Data":"c88e16acdada7ec1c339855e105e88c0cc9ff1de38d437d57442e12c499e26fd"} Jun 09 11:08:35.917338 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:35.917274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"0bc7ea2f8d216b49709892cb41e59bf618a7d1b6d7a269bca899f1a15e4c48ae"} Jun 09 11:08:36.925472 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.925400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"340ff99e9c1b9ebe58fead1765c77f5088864cc6da6466c61200f22db1ad026d"} Jun 09 11:08:36.925892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.925517 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"eb690d8683a1cecf8a12eb1bcb9199ce08ae98c73ddb91869f3c18a03c3e205a"} Jun 09 11:08:36.925892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.925586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"4e6b85bb9c8b5d3c3b7b2108a79a6fba486843aac794e094bfe886efaeb1be0f"} Jun 09 11:08:36.925892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.925631 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"85fe9082bf0bb0e943dcdde3f2c195bdcf6b05f19b33499cca1810fc123d6ed3"} Jun 09 11:08:36.927892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.927862 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"bacb13f50b6b9d58112fde7bf07b27f47c2706af39ba8331f09262abe3ba8dec"} Jun 09 11:08:36.929585 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.929523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" event={"ID":"6fdd4b06-f90f-4858-8716-534477fe901d","Type":"ContainerStarted","Data":"f6c688a0ee004c90c7ad6909c2296313e497669702e59f80d0365c8198348d4d"} Jun 09 11:08:36.929708 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.929689 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:36.931946 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.931360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" event={"ID":"213c7c8e-26bc-4038-be15-ed1bcc4480ae","Type":"ContainerStarted","Data":"4f2941d2731c0a713472e3f4705609f33bef27f300697144337314a8299504ef"} Jun 09 11:08:36.933862 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.933841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" event={"ID":"e266599b-4e54-4441-9ee8-50dfbf229f56","Type":"ContainerStarted","Data":"b13b394d578ec9f1a35aac009b90a3dc3701fcaebd9b9a504c817f1769384e2a"} Jun 09 11:08:36.933957 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.933889 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" event={"ID":"e266599b-4e54-4441-9ee8-50dfbf229f56","Type":"ContainerStarted","Data":"812c5a6f8b0080bfcac8e514b41f0bc215ec026770b4583c4226f2bea54731ca"} Jun 09 11:08:36.934009 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.933969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" event={"ID":"e266599b-4e54-4441-9ee8-50dfbf229f56","Type":"ContainerStarted","Data":"abeceb206c9fdd91dc1fd3ae24602854da8f2133d0a16d11b382803250357079"} Jun 09 11:08:36.935474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.935454 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" Jun 09 11:08:36.948796 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.948752 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-sxrd7" podStartSLOduration=2.981420837 podStartE2EDuration="4.948737817s" totalImagesPullingTime="1.96731698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:36.947512674 +0000 UTC m=+55.764202481" watchObservedRunningTime="2026-06-09 11:08:36.948737817 +0000 UTC m=+55.765427603" Jun 09 11:08:36.973169 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.973116 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-dd644d585-7gn4g" podStartSLOduration=1.316321756 podStartE2EDuration="3.973097208s" totalImagesPullingTime="2.656775452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:36.971767485 +0000 UTC m=+55.788457290" watchObservedRunningTime="2026-06-09 11:08:36.973097208 +0000 UTC m=+55.789787014" Jun 09 11:08:36.997640 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:36.997581 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" podStartSLOduration=1.999352888 podStartE2EDuration="4.997543586s" totalImagesPullingTime="2.998190698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:36.997005077 +0000 UTC m=+55.813694886" watchObservedRunningTime="2026-06-09 11:08:36.997543586 +0000 UTC m=+55.814233392" Jun 09 11:08:37.945004 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.944969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2b57d2dc-6df7-416a-96e0-59476b49d6c9","Type":"ContainerStarted","Data":"fbfe0ccc67f549491218f749f8f22b8d642b0175032ba3d0d13fc90ce710f3b2"} Jun 09 11:08:37.948599 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.948552 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"955b67b206917499f0f4e14a5ee43ecd625bdeb13c05ad5b89e0523764084387"} Jun 09 11:08:37.948735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.948635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"437ba9717fe0f0759b2d8315a7fe9d50ca3023e7a8e1c32acd3d94b0256c2c91"} Jun 09 11:08:37.948735 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.948677 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" event={"ID":"641bc884-e723-49c9-82c5-79b27c994999","Type":"ContainerStarted","Data":"cad9f658ffc8b21ac3dfa416fe41303ef14a5f1e8b92cb19c0d5db96562348ec"} Jun 09 11:08:37.974043 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.973993 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.960161779 podStartE2EDuration="8.973979843s" totalImagesPullingTime="4.000581697s" totalInitContainerRuntime="13.236367ms" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:37.972326338 +0000 UTC m=+56.789016143" watchObservedRunningTime="2026-06-09 11:08:37.973979843 +0000 UTC m=+56.790669647" Jun 09 11:08:37.999596 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:37.999528 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" podStartSLOduration=2.967123608 podStartE2EDuration="6.999508123s" totalImagesPullingTime="4.032384515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:37.998712246 +0000 UTC m=+56.815402053" watchObservedRunningTime="2026-06-09 11:08:37.999508123 +0000 UTC m=+56.816197929" Jun 09 11:08:38.954108 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:38.954080 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:39.961036 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:39.961010 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"726ce9a3f402f929655a73167ebf9466eeadb6ef2fd55863fe6b5fcaa5786b4a"} Jun 09 11:08:39.961389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:39.961091 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"d0b97c5d7876060c88a6eaac7e3cd936686047bf637a5624af56cb972df22132"} Jun 09 11:08:39.961389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:39.961146 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"b806102412c4a14450b02630286476b90082582072614bd34b36e60ab7cde1a4"} Jun 09 11:08:39.961389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:39.961187 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"8d4c6cdfaa1503cb680c9c65c36f2ea8e858211d82718ad7ade784b0cb00cf90"} Jun 09 11:08:39.961389 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:39.961225 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"dada59371972d5d08497a8116af29fa37d0da21ca804eadd48d6bee7b3b96096"} Jun 09 11:08:40.803708 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:40.803683 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78k77" Jun 09 11:08:40.967978 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:40.967947 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ef6e1b10-50c4-45d2-b53a-918ba01e65e7","Type":"ContainerStarted","Data":"32f7597a73d2a3b140e5eaacd6aae736112ac8c74794b65e1d84c3fb15384917"} Jun 09 11:08:41.002445 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:41.002399 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.288863009 podStartE2EDuration="7.002385052s" totalImagesPullingTime="3.676834735s" totalInitContainerRuntime="36.687308ms" isStatefulPod=false podCreationTimestamp="2026-06-09 11:08:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:41.000332277 +0000 UTC m=+59.817022103" watchObservedRunningTime="2026-06-09 11:08:41.002385052 +0000 UTC m=+59.819074857" Jun 09 11:08:44.674511 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:44.674482 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:08:44.967622 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:44.967531 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-56b7cb4894-kzg88" Jun 09 11:08:47.404176 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.404140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:47.407587 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.407557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 11:08:47.417061 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.417028 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75369ceb-252a-4d5e-b532-d20e91e04137-metrics-certs\") pod \"network-metrics-daemon-t6fzv\" (UID: \"75369ceb-252a-4d5e-b532-d20e91e04137\") " pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:47.505424 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.505387 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:47.508299 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.508279 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 11:08:47.518763 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.518735 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a67614cc-102f-483b-858c-f180e0540da9-original-pull-secret\") pod \"global-pull-secret-syncer-d2lzb\" (UID: \"a67614cc-102f-483b-858c-f180e0540da9\") " pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:47.606656 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.606618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:47.609671 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.609648 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 11:08:47.620065 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.620047 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 11:08:47.630435 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.630412 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8pk\" (UniqueName: \"kubernetes.io/projected/51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5-kube-api-access-qv8pk\") pod \"network-check-target-h928v\" (UID: \"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5\") " pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:47.663303 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.663238 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ft8nx\"" Jun 09 11:08:47.670772 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.670754 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:47.671215 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.671197 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-464pd\"" Jun 09 11:08:47.674776 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.674755 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d2lzb" Jun 09 11:08:47.679461 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.679436 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-t6fzv" Jun 09 11:08:47.837070 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:47.837042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-t6fzv"] Jun 09 11:08:47.840034 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:47.840008 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75369ceb_252a_4d5e_b532_d20e91e04137.slice/crio-f80a8949e1f63e22ee066961f050dd9f3ff1d19d6cb4e94ac6cce87d7bb167d4 WatchSource:0}: Error finding container f80a8949e1f63e22ee066961f050dd9f3ff1d19d6cb4e94ac6cce87d7bb167d4: Status 404 returned error can't find the container with id f80a8949e1f63e22ee066961f050dd9f3ff1d19d6cb4e94ac6cce87d7bb167d4 Jun 09 11:08:48.000100 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:48.000014 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6fzv" event={"ID":"75369ceb-252a-4d5e-b532-d20e91e04137","Type":"ContainerStarted","Data":"f80a8949e1f63e22ee066961f050dd9f3ff1d19d6cb4e94ac6cce87d7bb167d4"} Jun 09 11:08:48.064589 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:48.064547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d2lzb"] Jun 09 11:08:48.066621 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:48.066597 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67614cc_102f_483b_858c_f180e0540da9.slice/crio-c717559959981f0e16157f4da3a9a1b8f6b19d841419455d4e4218a5e9b101fb WatchSource:0}: Error finding container c717559959981f0e16157f4da3a9a1b8f6b19d841419455d4e4218a5e9b101fb: Status 404 returned error can't find the container with id c717559959981f0e16157f4da3a9a1b8f6b19d841419455d4e4218a5e9b101fb Jun 09 11:08:48.067165 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:48.067142 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h928v"] Jun 09 11:08:48.069446 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:08:48.069425 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51cdbcf1_5fae_4cb3_8c64_da0ab572a1d5.slice/crio-4e15bb5d095447f77d48d12f92321eafd1887a7b9a9255329be43880a5824c86 WatchSource:0}: Error finding container 4e15bb5d095447f77d48d12f92321eafd1887a7b9a9255329be43880a5824c86: Status 404 returned error can't find the container with id 4e15bb5d095447f77d48d12f92321eafd1887a7b9a9255329be43880a5824c86 Jun 09 11:08:49.007668 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:49.007598 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d2lzb" event={"ID":"a67614cc-102f-483b-858c-f180e0540da9","Type":"ContainerStarted","Data":"c717559959981f0e16157f4da3a9a1b8f6b19d841419455d4e4218a5e9b101fb"} Jun 09 11:08:49.009997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:49.009969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6fzv" event={"ID":"75369ceb-252a-4d5e-b532-d20e91e04137","Type":"ContainerStarted","Data":"ebb9cf079db58c61c7e91539e6067e36ac13054eff2f539e6f34ab36b36366d3"} Jun 09 11:08:49.010089 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:49.010021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-t6fzv" event={"ID":"75369ceb-252a-4d5e-b532-d20e91e04137","Type":"ContainerStarted","Data":"a07b11d2bc348dfcb0f36e563d6775189a60e4297e5f44834fa222136455429a"} Jun 09 11:08:49.012434 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:49.011805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h928v" event={"ID":"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5","Type":"ContainerStarted","Data":"4e15bb5d095447f77d48d12f92321eafd1887a7b9a9255329be43880a5824c86"} Jun 09 11:08:49.037166 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:49.037118 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-t6fzv" podStartSLOduration=67.123214872 podStartE2EDuration="1m8.037101754s" totalImagesPullingTime="913.886882ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:49.035768387 +0000 UTC m=+67.852458197" watchObservedRunningTime="2026-06-09 11:08:49.037101754 +0000 UTC m=+67.853791560" Jun 09 11:08:51.023204 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:51.023175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h928v" event={"ID":"51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5","Type":"ContainerStarted","Data":"dfd5f5050c178c97e644ccfd34d6727b17f2b1c93b0494b51b34b82d0d6d48a7"} Jun 09 11:08:51.023557 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:51.023374 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:08:51.044851 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:51.044807 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h928v" podStartSLOduration=66.171784945 podStartE2EDuration="1m9.044792831s" totalImagesPullingTime="2.873007886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:51.04346114 +0000 UTC m=+69.860150947" watchObservedRunningTime="2026-06-09 11:08:51.044792831 +0000 UTC m=+69.861482635" Jun 09 11:08:52.794489 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:52.794459 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:52.794931 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:52.794553 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:08:53.031439 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:53.031407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d2lzb" event={"ID":"a67614cc-102f-483b-858c-f180e0540da9","Type":"ContainerStarted","Data":"73c56aa21ed01c3bf9ffbe6fabc9e1840ffc3e07c1b9c05ccd76304d9d3c161a"} Jun 09 11:08:53.049369 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:08:53.049297 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d2lzb" podStartSLOduration=66.622348649 podStartE2EDuration="1m11.04928385s" totalImagesPullingTime="4.426935201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:08:53.047974362 +0000 UTC m=+71.864664184" watchObservedRunningTime="2026-06-09 11:08:53.04928385 +0000 UTC m=+71.865973654" Jun 09 11:09:12.799643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:12.799524 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:09:12.803201 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:12.803179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6ddfbcd75b-hch67" Jun 09 11:09:22.031145 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:22.030185 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h928v" Jun 09 11:09:38.601816 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:38.601784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:09:38.621137 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:38.621107 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:09:39.221720 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:09:39.221696 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 11:12:41.596631 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:12:41.596602 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:12:41.600025 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:12:41.600001 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:12:41.601238 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:12:41.601222 2560 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 11:17:41.619923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:17:41.619882 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:17:41.622313 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:17:41.622289 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:19:25.957194 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.957119 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-test-apply-4qrw5"] Jun 09 11:19:25.959302 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.959286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:25.962271 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.962253 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 09 11:19:25.962381 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.962276 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jun 09 11:19:25.963449 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.963430 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-h4ndf\"" Jun 09 11:19:25.963529 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.963432 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 09 11:19:25.968053 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:25.968036 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-4qrw5"] Jun 09 11:19:26.092369 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.092334 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlj4q\" (UniqueName: \"kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q\") pod \"feast-test-apply-4qrw5\" (UID: \"fbb8e01f-7341-4045-b359-d705b249feed\") " pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:26.193662 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.193627 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlj4q\" (UniqueName: \"kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q\") pod \"feast-test-apply-4qrw5\" (UID: \"fbb8e01f-7341-4045-b359-d705b249feed\") " pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:26.209841 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.209789 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlj4q\" (UniqueName: \"kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q\") pod \"feast-test-apply-4qrw5\" (UID: \"fbb8e01f-7341-4045-b359-d705b249feed\") " pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:26.279089 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.279065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:26.396525 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.396504 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-test-apply-4qrw5"] Jun 09 11:19:26.398649 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:19:26.398626 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfbb8e01f_7341_4045_b359_d705b249feed.slice/crio-e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c WatchSource:0}: Error finding container e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c: Status 404 returned error can't find the container with id e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c Jun 09 11:19:26.400733 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:26.400715 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 11:19:27.270095 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:27.270039 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerStarted","Data":"e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c"} Jun 09 11:19:32.292090 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:32.292060 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerStarted","Data":"da0723941560ceffacb5d458dde6af34b9624e7d180146385164f09a4df08ee8"} Jun 09 11:19:38.317035 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:38.317004 2560 generic.go:356] "Generic (PLEG): container finished" podID="fbb8e01f-7341-4045-b359-d705b249feed" containerID="da0723941560ceffacb5d458dde6af34b9624e7d180146385164f09a4df08ee8" exitCode=0 Jun 09 11:19:38.317387 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:38.317066 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerDied","Data":"da0723941560ceffacb5d458dde6af34b9624e7d180146385164f09a4df08ee8"} Jun 09 11:19:39.321823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:39.321791 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerStarted","Data":"af466dac7f4de94d2aabd816a3e418de1ad23874da6b01640bcf56c6c40d375d"} Jun 09 11:19:47.356741 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:47.356700 2560 generic.go:356] "Generic (PLEG): container finished" podID="fbb8e01f-7341-4045-b359-d705b249feed" containerID="af466dac7f4de94d2aabd816a3e418de1ad23874da6b01640bcf56c6c40d375d" exitCode=0 Jun 09 11:19:47.357199 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:47.356764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerDied","Data":"af466dac7f4de94d2aabd816a3e418de1ad23874da6b01640bcf56c6c40d375d"} Jun 09 11:19:48.489725 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:48.489697 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:48.571477 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:48.571451 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlj4q\" (UniqueName: \"kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q\") pod \"fbb8e01f-7341-4045-b359-d705b249feed\" (UID: \"fbb8e01f-7341-4045-b359-d705b249feed\") " Jun 09 11:19:48.573463 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:48.573440 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q" (OuterVolumeSpecName: "kube-api-access-vlj4q") pod "fbb8e01f-7341-4045-b359-d705b249feed" (UID: "fbb8e01f-7341-4045-b359-d705b249feed"). InnerVolumeSpecName "kube-api-access-vlj4q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:19:48.672070 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:48.672008 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlj4q\" (UniqueName: \"kubernetes.io/projected/fbb8e01f-7341-4045-b359-d705b249feed-kube-api-access-vlj4q\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:19:49.365474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:49.365438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-test-apply-4qrw5" event={"ID":"fbb8e01f-7341-4045-b359-d705b249feed","Type":"ContainerDied","Data":"e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c"} Jun 09 11:19:49.365474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:49.365464 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-test-apply-4qrw5" Jun 09 11:19:49.365474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:49.365478 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e056399b3dc1a3a27d42434f9e3acf26e0853913a7685b88a103bc02ade2cd0c" Jun 09 11:19:49.575362 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:49.575335 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-4qrw5_fbb8e01f-7341-4045-b359-d705b249feed/feast-0/0.log" Jun 09 11:19:49.581813 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:19:49.581796 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/test-ns-feast_feast-test-apply-4qrw5_fbb8e01f-7341-4045-b359-d705b249feed/feast-1/0.log" Jun 09 11:20:24.484340 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:20:24.484305 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-test-apply-4qrw5"] Jun 09 11:20:24.490316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:20:24.490287 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-test-apply-4qrw5"] Jun 09 11:20:25.649890 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:20:25.649860 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbb8e01f-7341-4045-b359-d705b249feed" path="/var/lib/kubelet/pods/fbb8e01f-7341-4045-b359-d705b249feed/volumes" Jun 09 11:22:41.647639 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:22:41.647611 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:22:41.648699 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:22:41.648678 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:25:12.553831 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.553796 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:25:12.554240 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.554231 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb8e01f-7341-4045-b359-d705b249feed" containerName="feast-1" Jun 09 11:25:12.554299 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.554245 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb8e01f-7341-4045-b359-d705b249feed" containerName="feast-1" Jun 09 11:25:12.554299 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.554260 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb8e01f-7341-4045-b359-d705b249feed" containerName="feast-0" Jun 09 11:25:12.554299 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.554269 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb8e01f-7341-4045-b359-d705b249feed" containerName="feast-0" Jun 09 11:25:12.554460 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.554340 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbb8e01f-7341-4045-b359-d705b249feed" containerName="feast-1" Jun 09 11:25:12.557141 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.557124 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:12.560282 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.560253 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jun 09 11:25:12.560368 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.560295 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wtxb9\"" Jun 09 11:25:12.560940 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.560927 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jun 09 11:25:12.568139 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.568119 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:25:12.709039 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.708992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbjvg\" (UniqueName: \"kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg\") pod \"feast-operator-controller-manager-6d54c7db9d-sk7q7\" (UID: \"df719a2d-f019-4fad-8ccd-f92a17befebf\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:12.809519 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.809455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbjvg\" (UniqueName: \"kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg\") pod \"feast-operator-controller-manager-6d54c7db9d-sk7q7\" (UID: \"df719a2d-f019-4fad-8ccd-f92a17befebf\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:12.819617 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.819595 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbjvg\" (UniqueName: \"kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg\") pod \"feast-operator-controller-manager-6d54c7db9d-sk7q7\" (UID: \"df719a2d-f019-4fad-8ccd-f92a17befebf\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:12.865983 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.865958 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:12.988833 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.988810 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:25:12.991243 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:25:12.991206 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf719a2d_f019_4fad_8ccd_f92a17befebf.slice/crio-3682bb194110b86a494fac1e6709b4dd500c0c20ec9a47daeea9cdef4199e88c WatchSource:0}: Error finding container 3682bb194110b86a494fac1e6709b4dd500c0c20ec9a47daeea9cdef4199e88c: Status 404 returned error can't find the container with id 3682bb194110b86a494fac1e6709b4dd500c0c20ec9a47daeea9cdef4199e88c Jun 09 11:25:12.993057 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:12.993041 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 11:25:13.560670 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:13.560629 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" event={"ID":"df719a2d-f019-4fad-8ccd-f92a17befebf","Type":"ContainerStarted","Data":"3682bb194110b86a494fac1e6709b4dd500c0c20ec9a47daeea9cdef4199e88c"} Jun 09 11:25:17.581281 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:17.581253 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" event={"ID":"df719a2d-f019-4fad-8ccd-f92a17befebf","Type":"ContainerStarted","Data":"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0"} Jun 09 11:25:17.581665 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:17.581345 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:17.603759 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:17.603717 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" podStartSLOduration=1.96806704 podStartE2EDuration="5.603703884s" totalImagesPullingTime="3.635636844s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:25:17.60228243 +0000 UTC m=+1056.418972236" watchObservedRunningTime="2026-06-09 11:25:17.603703884 +0000 UTC m=+1056.420393688" Jun 09 11:25:28.587208 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:28.587131 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:25:31.351093 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.351056 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:25:31.354698 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.354677 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.357271 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357249 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 09 11:25:31.357368 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357354 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 09 11:25:31.357432 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357375 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wzfxx\"" Jun 09 11:25:31.357432 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357390 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 09 11:25:31.357513 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357498 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 09 11:25:31.357557 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.357551 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 09 11:25:31.358457 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.358432 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 09 11:25:31.367618 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.367599 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:25:31.447455 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447432 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.447552 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.447552 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.447552 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447539 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4sgj\" (UniqueName: \"kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.447709 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.447709 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.447643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548458 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548560 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548461 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548560 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548493 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548560 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548744 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548744 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548623 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4sgj\" (UniqueName: \"kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.548842 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.548817 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.550942 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.550922 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.551029 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.550972 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.551127 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.551109 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.551184 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.551165 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.557154 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.557136 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4sgj\" (UniqueName: \"kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj\") pod \"feast-simple-feast-setup-75c64c888b-x766h\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.678360 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.678288 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:25:31.810956 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:31.810927 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:25:31.814021 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:25:31.813998 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf6be8c6e_c63a_4e52_a2ff_40beee4119a4.slice/crio-3dcba754725233237806f46478da0f1346c1aab2ff9c29bc820c39be333dcda1 WatchSource:0}: Error finding container 3dcba754725233237806f46478da0f1346c1aab2ff9c29bc820c39be333dcda1: Status 404 returned error can't find the container with id 3dcba754725233237806f46478da0f1346c1aab2ff9c29bc820c39be333dcda1 Jun 09 11:25:32.643618 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:32.643552 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"3dcba754725233237806f46478da0f1346c1aab2ff9c29bc820c39be333dcda1"} Jun 09 11:25:41.668023 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:41.667991 2560 scope.go:117] "RemoveContainer" containerID="af466dac7f4de94d2aabd816a3e418de1ad23874da6b01640bcf56c6c40d375d" Jun 09 11:25:46.589896 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:46.589859 2560 scope.go:117] "RemoveContainer" containerID="da0723941560ceffacb5d458dde6af34b9624e7d180146385164f09a4df08ee8" Jun 09 11:25:47.709806 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:47.709771 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"c4688b9deb1d5b4d584701894727dd0c52db2cf88b3dde4f51c24d1ccb5e976d"} Jun 09 11:25:51.726511 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:51.726477 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="c4688b9deb1d5b4d584701894727dd0c52db2cf88b3dde4f51c24d1ccb5e976d" exitCode=0 Jun 09 11:25:51.726974 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:51.726513 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"c4688b9deb1d5b4d584701894727dd0c52db2cf88b3dde4f51c24d1ccb5e976d"} Jun 09 11:25:52.731857 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:52.731827 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"3de86396c2dc96bbc8a2159895770f6e6bc05f4c20c5fc930fbce11548e7a9df"} Jun 09 11:25:57.751579 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:57.751534 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="3de86396c2dc96bbc8a2159895770f6e6bc05f4c20c5fc930fbce11548e7a9df" exitCode=0 Jun 09 11:25:57.751938 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:57.751609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"3de86396c2dc96bbc8a2159895770f6e6bc05f4c20c5fc930fbce11548e7a9df"} Jun 09 11:25:58.761145 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:58.761108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"9f273ad676a4a28c1d9f8d871e5f281759cbab88fdbacfd9c74a8dad840a3351"} Jun 09 11:25:58.761634 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:58.761164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"d0430f463ba16fae58fe472d1a768500125ab378d07a48d81c991b56942bf556"} Jun 09 11:25:58.761634 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:58.761184 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"1e538dd58060f927824e5a32fe595a54d2a92fda1ffb70f26d14b2213a576dec"} Jun 09 11:25:58.761634 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:58.761202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerStarted","Data":"dc9291ae53ba921f12a45dca4d3526faeaa2a43a2c9860ed2a8e812944ebeb9c"} Jun 09 11:25:58.788274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:25:58.788212 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podStartSLOduration=12.929239727 podStartE2EDuration="27.788192853s" totalImagesPullingTime="14.858953126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:25:58.785003095 +0000 UTC m=+1097.601692900" watchObservedRunningTime="2026-06-09 11:25:58.788192853 +0000 UTC m=+1097.604882660" Jun 09 11:26:01.679351 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679316 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.679899 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679875 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679908 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679926 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679943 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679961 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679976 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.680034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.679992 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:01.681471 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.681426 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.18:6571: connect: connection refused" Jun 09 11:26:01.681471 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.681457 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Jun 09 11:26:01.681666 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.681507 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Jun 09 11:26:01.681666 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:01.681627 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.18:8816: connect: connection refused" Jun 09 11:26:04.680036 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.680005 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.680691 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.680670 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.680840 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.680829 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.681376 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.681354 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.681732 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.681710 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.681814 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.681774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.684400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.684380 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:26:04.793481 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:26:04.793452 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:27:06.584557 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.584478 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:27:06.585097 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.584760 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="registry" containerID="cri-o://dc9291ae53ba921f12a45dca4d3526faeaa2a43a2c9860ed2a8e812944ebeb9c" gracePeriod=30 Jun 09 11:27:06.585097 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.584809 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" containerID="cri-o://9f273ad676a4a28c1d9f8d871e5f281759cbab88fdbacfd9c74a8dad840a3351" gracePeriod=30 Jun 09 11:27:06.585097 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.584797 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="offline" containerID="cri-o://d0430f463ba16fae58fe472d1a768500125ab378d07a48d81c991b56942bf556" gracePeriod=30 Jun 09 11:27:06.585097 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.584832 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" containerID="cri-o://1e538dd58060f927824e5a32fe595a54d2a92fda1ffb70f26d14b2213a576dec" gracePeriod=30 Jun 09 11:27:06.881130 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.881050 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:27:06.884789 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.884769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.887487 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.887464 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-z56k8\"" Jun 09 11:27:06.898134 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.898111 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:27:06.979677 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.979846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979690 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.979846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979747 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.979846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979797 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.979846 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:06.979998 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:06.979855 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds2sb\" (UniqueName: \"kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081051 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081129 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081161 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds2sb\" (UniqueName: \"kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.081489 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:27:07.081264 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jun 09 11:27:07.081489 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:27:07.081349 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls podName:d511c575-6885-404b-adef-8f06227258bd nodeName:}" failed. No retries permitted until 2026-06-09 11:27:07.581327673 +0000 UTC m=+1166.398017471 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls") pod "feast-simple-feast-setup-75c64c888b-wrb56" (UID: "d511c575-6885-404b-adef-8f06227258bd") : secret "feast-simple-feast-setup-offline-tls" not found Jun 09 11:27:07.081489 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:27:07.081354 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jun 09 11:27:07.081489 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:27:07.081411 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls podName:d511c575-6885-404b-adef-8f06227258bd nodeName:}" failed. No retries permitted until 2026-06-09 11:27:07.581396119 +0000 UTC m=+1166.398085922 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls") pod "feast-simple-feast-setup-75c64c888b-wrb56" (UID: "d511c575-6885-404b-adef-8f06227258bd") : secret "feast-simple-feast-setup-online-tls" not found Jun 09 11:27:07.081489 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.081478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.083729 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.083710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.083833 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.083757 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.092035 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.091990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds2sb\" (UniqueName: \"kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.586353 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.586327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.586728 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.586366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.588826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.588797 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.588924 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.588809 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-wrb56\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.794940 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.794918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:07.923517 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:07.923492 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:27:07.925091 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:27:07.925069 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd511c575_6885_404b_adef_8f06227258bd.slice/crio-a538f99c0d3447f2af46ec91186a6e79bcb4fb4db9372cfc07c30221f2886eec WatchSource:0}: Error finding container a538f99c0d3447f2af46ec91186a6e79bcb4fb4db9372cfc07c30221f2886eec: Status 404 returned error can't find the container with id a538f99c0d3447f2af46ec91186a6e79bcb4fb4db9372cfc07c30221f2886eec Jun 09 11:27:08.033559 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.033518 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"5cbaeca55cf799d1ddcaabf0160dbe2d12969ef3ada8deed059b6f0e3d2594f0"} Jun 09 11:27:08.033703 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.033588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"a538f99c0d3447f2af46ec91186a6e79bcb4fb4db9372cfc07c30221f2886eec"} Jun 09 11:27:08.036507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.036486 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="9f273ad676a4a28c1d9f8d871e5f281759cbab88fdbacfd9c74a8dad840a3351" exitCode=0 Jun 09 11:27:08.036507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.036504 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="1e538dd58060f927824e5a32fe595a54d2a92fda1ffb70f26d14b2213a576dec" exitCode=0 Jun 09 11:27:08.036702 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.036523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"9f273ad676a4a28c1d9f8d871e5f281759cbab88fdbacfd9c74a8dad840a3351"} Jun 09 11:27:08.036702 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:08.036545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"1e538dd58060f927824e5a32fe595a54d2a92fda1ffb70f26d14b2213a576dec"} Jun 09 11:27:13.061699 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:13.061665 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="5cbaeca55cf799d1ddcaabf0160dbe2d12969ef3ada8deed059b6f0e3d2594f0" exitCode=0 Jun 09 11:27:13.062071 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:13.061737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"5cbaeca55cf799d1ddcaabf0160dbe2d12969ef3ada8deed059b6f0e3d2594f0"} Jun 09 11:27:14.067243 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:14.067204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"a9fff8ee687e580ee1ac1f06ec8564a3ef2f0593d984fdc6263b61773be214f8"} Jun 09 11:27:14.682051 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:14.681981 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Jun 09 11:27:14.790886 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:14.790849 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Jun 09 11:27:19.093901 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:19.093866 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="a9fff8ee687e580ee1ac1f06ec8564a3ef2f0593d984fdc6263b61773be214f8" exitCode=0 Jun 09 11:27:19.093901 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:19.093905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"a9fff8ee687e580ee1ac1f06ec8564a3ef2f0593d984fdc6263b61773be214f8"} Jun 09 11:27:20.103955 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:20.103917 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"6436d0556ccea73860c0f46a5c88134933ab073c818d1772b140aa7c211b1c0a"} Jun 09 11:27:20.104418 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:20.103978 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"67afbf92059f8d573b7914d777d0d9f496ccf08177ee6efc6fd826de0acb6b1c"} Jun 09 11:27:20.104418 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:20.104000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"a89098d65294973e7e01923ca69e4c1eaaf8f8b42dc5b95c89a4df0b2fd74fa0"} Jun 09 11:27:20.104418 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:20.104020 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerStarted","Data":"7b8885b245c2f368f6dcee2dfa5e3472b2ee706beae7f6997483f263718e5102"} Jun 09 11:27:20.131886 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:20.131823 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podStartSLOduration=14.131802701 podStartE2EDuration="14.131802701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:27:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:27:20.129720786 +0000 UTC m=+1178.946410593" watchObservedRunningTime="2026-06-09 11:27:20.131802701 +0000 UTC m=+1178.948492506" Jun 09 11:27:22.796119 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.795190 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:22.796119 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.795247 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:22.796119 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.795268 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:22.796119 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.795285 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:22.797341 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.797255 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.19:8816: connect: connection refused" Jun 09 11:27:22.797502 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.797321 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jun 09 11:27:22.797597 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.797339 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.19:6571: connect: connection refused" Jun 09 11:27:22.797597 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:22.797372 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jun 09 11:27:24.681516 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:24.681470 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Jun 09 11:27:24.791410 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:24.791381 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Jun 09 11:27:25.796363 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.796334 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.796796 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.796453 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.796796 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.796688 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797047 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.796967 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797047 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.797006 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797047 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.797026 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797356 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.797336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797532 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.797510 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.797634 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.797590 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:25.800632 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:25.800616 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:26.138661 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:26.138596 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:26.141245 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:26.141224 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:27:34.681836 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:34.681784 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.18:8443: connect: connection refused" Jun 09 11:27:34.682260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:34.681939 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:27:34.791658 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:34.791623 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.18:6567/health\": dial tcp 10.134.0.18:6567: connect: connection refused" Jun 09 11:27:34.791801 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:34.791706 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:27:37.184180 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.184150 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="d0430f463ba16fae58fe472d1a768500125ab378d07a48d81c991b56942bf556" exitCode=137 Jun 09 11:27:37.184180 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.184175 2560 generic.go:356] "Generic (PLEG): container finished" podID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerID="dc9291ae53ba921f12a45dca4d3526faeaa2a43a2c9860ed2a8e812944ebeb9c" exitCode=137 Jun 09 11:27:37.184523 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.184222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"d0430f463ba16fae58fe472d1a768500125ab378d07a48d81c991b56942bf556"} Jun 09 11:27:37.184523 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.184258 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"dc9291ae53ba921f12a45dca4d3526faeaa2a43a2c9860ed2a8e812944ebeb9c"} Jun 09 11:27:37.229994 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.229976 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:27:37.340668 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.340640 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.340873 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.340676 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.340873 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.340706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.340873 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.340852 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4sgj\" (UniqueName: \"kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.341030 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.340922 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.341084 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.341027 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls\") pod \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\" (UID: \"f6be8c6e-c63a-4e52-a2ff-40beee4119a4\") " Jun 09 11:27:37.341237 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.341213 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data" (OuterVolumeSpecName: "feast-data") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:27:37.341366 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.341352 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:37.343105 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.343077 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:27:37.343105 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.343085 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj" (OuterVolumeSpecName: "kube-api-access-j4sgj") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "kube-api-access-j4sgj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:27:37.343248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.343122 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:27:37.343248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.343135 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:27:37.343248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.343168 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls" (OuterVolumeSpecName: "online-tls") pod "f6be8c6e-c63a-4e52-a2ff-40beee4119a4" (UID: "f6be8c6e-c63a-4e52-a2ff-40beee4119a4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:27:37.442700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.442633 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4sgj\" (UniqueName: \"kubernetes.io/projected/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-kube-api-access-j4sgj\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:37.442700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.442656 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:37.442700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.442665 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:37.442700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.442674 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:37.442700 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:37.442681 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f6be8c6e-c63a-4e52-a2ff-40beee4119a4-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:27:38.191005 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.190981 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" Jun 09 11:27:38.191369 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.190980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h" event={"ID":"f6be8c6e-c63a-4e52-a2ff-40beee4119a4","Type":"ContainerDied","Data":"3dcba754725233237806f46478da0f1346c1aab2ff9c29bc820c39be333dcda1"} Jun 09 11:27:38.191369 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.191111 2560 scope.go:117] "RemoveContainer" containerID="9f273ad676a4a28c1d9f8d871e5f281759cbab88fdbacfd9c74a8dad840a3351" Jun 09 11:27:38.199090 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.199073 2560 scope.go:117] "RemoveContainer" containerID="d0430f463ba16fae58fe472d1a768500125ab378d07a48d81c991b56942bf556" Jun 09 11:27:38.206042 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.206028 2560 scope.go:117] "RemoveContainer" containerID="1e538dd58060f927824e5a32fe595a54d2a92fda1ffb70f26d14b2213a576dec" Jun 09 11:27:38.209922 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.209884 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:27:38.213545 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.213528 2560 scope.go:117] "RemoveContainer" containerID="dc9291ae53ba921f12a45dca4d3526faeaa2a43a2c9860ed2a8e812944ebeb9c" Jun 09 11:27:38.215489 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.215473 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-x766h"] Jun 09 11:27:38.220613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.220597 2560 scope.go:117] "RemoveContainer" containerID="3de86396c2dc96bbc8a2159895770f6e6bc05f4c20c5fc930fbce11548e7a9df" Jun 09 11:27:38.227077 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:38.227058 2560 scope.go:117] "RemoveContainer" containerID="c4688b9deb1d5b4d584701894727dd0c52db2cf88b3dde4f51c24d1ccb5e976d" Jun 09 11:27:39.650882 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:39.650844 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" path="/var/lib/kubelet/pods/f6be8c6e-c63a-4e52-a2ff-40beee4119a4/volumes" Jun 09 11:27:41.673353 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:41.673322 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:27:41.673812 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:27:41.673763 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:28:28.832132 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832059 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832357 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="offline" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832367 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="offline" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832378 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832384 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832390 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="feast-init" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832399 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="feast-init" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832409 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="registry" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832417 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="registry" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832435 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832444 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832457 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="feast-apply" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832462 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="feast-apply" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832506 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="offline" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832514 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="ui" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832520 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="registry" Jun 09 11:28:28.832546 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.832528 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6be8c6e-c63a-4e52-a2ff-40beee4119a4" containerName="online" Jun 09 11:28:28.835964 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.835948 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.838919 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.838898 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 09 11:28:28.839035 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.838944 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 09 11:28:28.839107 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.839051 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 09 11:28:28.839183 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.839165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-jt6fk\"" Jun 09 11:28:28.839237 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.839181 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 09 11:28:28.839337 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.839322 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 09 11:28:28.839426 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.839414 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 09 11:28:28.846128 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.846109 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:28:28.918466 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918445 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.918613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.918613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918506 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.918613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5qst\" (UniqueName: \"kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.918613 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:28.918755 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:28.918619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.018962 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.018908 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.018962 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.018940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.018965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5qst\" (UniqueName: \"kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.019016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.019043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019136 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.019082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019396 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.019377 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.019764 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.019743 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.021336 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.021319 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.021423 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.021356 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.021423 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.021406 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.027740 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.027719 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5qst\" (UniqueName: \"kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst\") pod \"feast-simple-feast-remote-setup-669bcf45fc-vcfnr\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.145449 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.145390 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:29.272755 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.272733 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:28:29.274662 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:28:29.274635 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd13561b8_bbb2_42d6_b2ea_495bd1774deb.slice/crio-d253b23e431fdaca8058455631143539387daba722777941dc557d0f5d63669d WatchSource:0}: Error finding container d253b23e431fdaca8058455631143539387daba722777941dc557d0f5d63669d: Status 404 returned error can't find the container with id d253b23e431fdaca8058455631143539387daba722777941dc557d0f5d63669d Jun 09 11:28:29.388534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.388502 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3"} Jun 09 11:28:29.388674 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:29.388550 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"d253b23e431fdaca8058455631143539387daba722777941dc557d0f5d63669d"} Jun 09 11:28:34.410739 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:34.410699 2560 generic.go:356] "Generic (PLEG): container finished" podID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerID="54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3" exitCode=0 Jun 09 11:28:34.411076 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:34.410749 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3"} Jun 09 11:28:35.417046 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:35.417012 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5"} Jun 09 11:28:40.437919 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:40.437889 2560 generic.go:356] "Generic (PLEG): container finished" podID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerID="65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5" exitCode=0 Jun 09 11:28:40.438265 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:40.437925 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5"} Jun 09 11:28:41.447082 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:41.447045 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d"} Jun 09 11:28:41.447647 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:41.447626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186"} Jun 09 11:28:41.447726 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:41.447659 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerStarted","Data":"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636"} Jun 09 11:28:41.475398 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:41.475335 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podStartSLOduration=13.475315509 podStartE2EDuration="13.475315509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:28:41.472750799 +0000 UTC m=+1260.289440605" watchObservedRunningTime="2026-06-09 11:28:41.475315509 +0000 UTC m=+1260.292005316" Jun 09 11:28:44.146453 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.146419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:44.146926 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.146558 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:44.146926 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.146605 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:44.148274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.148248 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.20:8816: connect: connection refused" Jun 09 11:28:44.148387 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.148267 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jun 09 11:28:44.148387 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:44.148284 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jun 09 11:28:47.146715 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.146679 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.147102 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.146835 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.147144 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.147112 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.147205 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.147142 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.147487 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.147470 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.147662 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.147643 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.150887 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.150871 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.475334 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.475245 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:28:47.478104 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:28:47.478086 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:29:50.128140 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.128109 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:29:50.128743 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.128326 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" containerID="cri-o://8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636" gracePeriod=30 Jun 09 11:29:50.128743 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.128390 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="offline" containerID="cri-o://55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186" gracePeriod=30 Jun 09 11:29:50.128743 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.128399 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" containerID="cri-o://c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d" gracePeriod=30 Jun 09 11:29:50.282221 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.282189 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:29:50.282539 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.282469 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="registry" containerID="cri-o://7b8885b245c2f368f6dcee2dfa5e3472b2ee706beae7f6997483f263718e5102" gracePeriod=30 Jun 09 11:29:50.282539 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.282494 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="offline" containerID="cri-o://67afbf92059f8d573b7914d777d0d9f496ccf08177ee6efc6fd826de0acb6b1c" gracePeriod=30 Jun 09 11:29:50.282731 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.282514 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" containerID="cri-o://a89098d65294973e7e01923ca69e4c1eaaf8f8b42dc5b95c89a4df0b2fd74fa0" gracePeriod=30 Jun 09 11:29:50.282731 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.282521 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" containerID="cri-o://6436d0556ccea73860c0f46a5c88134933ab073c818d1772b140aa7c211b1c0a" gracePeriod=30 Jun 09 11:29:50.732077 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.732044 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="6436d0556ccea73860c0f46a5c88134933ab073c818d1772b140aa7c211b1c0a" exitCode=0 Jun 09 11:29:50.732250 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.732113 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"6436d0556ccea73860c0f46a5c88134933ab073c818d1772b140aa7c211b1c0a"} Jun 09 11:29:50.734951 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.734924 2560 generic.go:356] "Generic (PLEG): container finished" podID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerID="c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d" exitCode=0 Jun 09 11:29:50.735082 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:50.734961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d"} Jun 09 11:29:51.741515 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:51.741486 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="a89098d65294973e7e01923ca69e4c1eaaf8f8b42dc5b95c89a4df0b2fd74fa0" exitCode=0 Jun 09 11:29:51.741909 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:51.741557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"a89098d65294973e7e01923ca69e4c1eaaf8f8b42dc5b95c89a4df0b2fd74fa0"} Jun 09 11:29:51.743885 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:51.743861 2560 generic.go:356] "Generic (PLEG): container finished" podID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerID="8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636" exitCode=0 Jun 09 11:29:51.743990 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:51.743897 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636"} Jun 09 11:29:55.797559 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:55.797466 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jun 09 11:29:56.139171 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:56.139093 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jun 09 11:29:57.147205 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:57.147167 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jun 09 11:29:57.475969 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:29:57.475884 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jun 09 11:30:05.797375 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:05.797336 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jun 09 11:30:06.139467 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:06.139382 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jun 09 11:30:07.148219 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:07.148177 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jun 09 11:30:07.475861 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:07.475764 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jun 09 11:30:15.797256 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:15.797216 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.19:8443: connect: connection refused" Jun 09 11:30:15.797705 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:15.797330 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:30:16.138750 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:16.138669 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.19:6567/health\": dial tcp 10.134.0.19:6567: connect: connection refused" Jun 09 11:30:16.138885 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:16.138771 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:30:17.147473 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:17.147441 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.20:8443: connect: connection refused" Jun 09 11:30:17.147875 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:17.147555 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:30:17.476355 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:17.476271 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.20:6567/health\": dial tcp 10.134.0.20:6567: connect: connection refused" Jun 09 11:30:17.476494 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:17.476383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:30:20.767861 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.767833 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:30:20.855783 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.855753 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="67afbf92059f8d573b7914d777d0d9f496ccf08177ee6efc6fd826de0acb6b1c" exitCode=137 Jun 09 11:30:20.855783 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.855779 2560 generic.go:356] "Generic (PLEG): container finished" podID="d511c575-6885-404b-adef-8f06227258bd" containerID="7b8885b245c2f368f6dcee2dfa5e3472b2ee706beae7f6997483f263718e5102" exitCode=137 Jun 09 11:30:20.855968 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.855821 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"67afbf92059f8d573b7914d777d0d9f496ccf08177ee6efc6fd826de0acb6b1c"} Jun 09 11:30:20.855968 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.855873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"7b8885b245c2f368f6dcee2dfa5e3472b2ee706beae7f6997483f263718e5102"} Jun 09 11:30:20.858153 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.858132 2560 generic.go:356] "Generic (PLEG): container finished" podID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerID="55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186" exitCode=137 Jun 09 11:30:20.858260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.858192 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186"} Jun 09 11:30:20.858260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.858211 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" Jun 09 11:30:20.858260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.858222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr" event={"ID":"d13561b8-bbb2-42d6-b2ea-495bd1774deb","Type":"ContainerDied","Data":"d253b23e431fdaca8058455631143539387daba722777941dc557d0f5d63669d"} Jun 09 11:30:20.858260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.858240 2560 scope.go:117] "RemoveContainer" containerID="c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d" Jun 09 11:30:20.859212 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859194 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859285 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859267 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859289 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5qst\" (UniqueName: \"kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859329 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859447 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859357 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859447 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859384 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls\") pod \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\" (UID: \"d13561b8-bbb2-42d6-b2ea-495bd1774deb\") " Jun 09 11:30:20.859928 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859892 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data" (OuterVolumeSpecName: "feast-data") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:30:20.859928 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.859915 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 11:30:20.863791 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.863766 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:20.863918 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.863897 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:20.863918 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.863907 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst" (OuterVolumeSpecName: "kube-api-access-t5qst") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "kube-api-access-t5qst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:30:20.863993 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.863922 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls" (OuterVolumeSpecName: "online-tls") pod "d13561b8-bbb2-42d6-b2ea-495bd1774deb" (UID: "d13561b8-bbb2-42d6-b2ea-495bd1774deb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:20.871841 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.871822 2560 scope.go:117] "RemoveContainer" containerID="55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186" Jun 09 11:30:20.879134 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.879117 2560 scope.go:117] "RemoveContainer" containerID="8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636" Jun 09 11:30:20.886717 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.886695 2560 scope.go:117] "RemoveContainer" containerID="65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5" Jun 09 11:30:20.895150 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.895134 2560 scope.go:117] "RemoveContainer" containerID="54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3" Jun 09 11:30:20.903168 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.903144 2560 scope.go:117] "RemoveContainer" containerID="c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d" Jun 09 11:30:20.903432 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:20.903414 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d\": container with ID starting with c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d not found: ID does not exist" containerID="c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d" Jun 09 11:30:20.903486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.903439 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d"} err="failed to get container status \"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d\": rpc error: code = NotFound desc = could not find container \"c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d\": container with ID starting with c078f7d00ad3e50a33fd346bf219888f3a1d693daae00fafd93761b5dda7bf0d not found: ID does not exist" Jun 09 11:30:20.903486 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.903470 2560 scope.go:117] "RemoveContainer" containerID="55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186" Jun 09 11:30:20.903771 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:20.903753 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186\": container with ID starting with 55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186 not found: ID does not exist" containerID="55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186" Jun 09 11:30:20.903820 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.903776 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186"} err="failed to get container status \"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186\": rpc error: code = NotFound desc = could not find container \"55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186\": container with ID starting with 55d92aed52fc2ccecb9b3323e7baebc0ad9dc253f9a0349beaf8609527630186 not found: ID does not exist" Jun 09 11:30:20.903820 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.903797 2560 scope.go:117] "RemoveContainer" containerID="8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636" Jun 09 11:30:20.904020 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:20.904005 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636\": container with ID starting with 8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636 not found: ID does not exist" containerID="8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636" Jun 09 11:30:20.904068 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.904022 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636"} err="failed to get container status \"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636\": rpc error: code = NotFound desc = could not find container \"8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636\": container with ID starting with 8ae3a6ba9a0a26d4a91c6bcc2048a0ee66dab59cbcb18088380382f6981d9636 not found: ID does not exist" Jun 09 11:30:20.904068 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.904047 2560 scope.go:117] "RemoveContainer" containerID="65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5" Jun 09 11:30:20.904277 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:20.904259 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5\": container with ID starting with 65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5 not found: ID does not exist" containerID="65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5" Jun 09 11:30:20.904326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.904283 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5"} err="failed to get container status \"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5\": rpc error: code = NotFound desc = could not find container \"65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5\": container with ID starting with 65dea5e577132b3a3fd9db5da2175672e29f48a0ce89a077b92a1407f5897cc5 not found: ID does not exist" Jun 09 11:30:20.904326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.904299 2560 scope.go:117] "RemoveContainer" containerID="54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3" Jun 09 11:30:20.904522 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:20.904507 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3\": container with ID starting with 54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3 not found: ID does not exist" containerID="54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3" Jun 09 11:30:20.904688 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.904524 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3"} err="failed to get container status \"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3\": rpc error: code = NotFound desc = could not find container \"54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3\": container with ID starting with 54775e2b13f9c037ed84c233858063ba42ca3344a1dcf9fc06e34d1582dfb1f3 not found: ID does not exist" Jun 09 11:30:20.924744 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.924722 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:30:20.960436 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960412 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/d13561b8-bbb2-42d6-b2ea-495bd1774deb-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:20.960436 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960438 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:20.960588 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960449 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:20.960588 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960458 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5qst\" (UniqueName: \"kubernetes.io/projected/d13561b8-bbb2-42d6-b2ea-495bd1774deb-kube-api-access-t5qst\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:20.960588 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960467 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d13561b8-bbb2-42d6-b2ea-495bd1774deb-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:20.960588 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:20.960475 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d13561b8-bbb2-42d6-b2ea-495bd1774deb-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.060781 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060750 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.060781 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060786 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ds2sb\" (UniqueName: \"kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.060953 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060809 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.060953 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060832 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.060953 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060864 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.060953 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.060888 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls\") pod \"d511c575-6885-404b-adef-8f06227258bd\" (UID: \"d511c575-6885-404b-adef-8f06227258bd\") " Jun 09 11:30:21.061337 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.061308 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data" (OuterVolumeSpecName: "feast-data") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:30:21.063066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.063044 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:21.063370 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.063351 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:21.063370 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.063356 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb" (OuterVolumeSpecName: "kube-api-access-ds2sb") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "kube-api-access-ds2sb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:30:21.063467 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.063374 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:21.063467 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.063384 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls" (OuterVolumeSpecName: "online-tls") pod "d511c575-6885-404b-adef-8f06227258bd" (UID: "d511c575-6885-404b-adef-8f06227258bd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162300 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d511c575-6885-404b-adef-8f06227258bd-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162320 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds2sb\" (UniqueName: \"kubernetes.io/projected/d511c575-6885-404b-adef-8f06227258bd-kube-api-access-ds2sb\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162328 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162336 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162344 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.162347 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.162351 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d511c575-6885-404b-adef-8f06227258bd-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:21.179983 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.179960 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:30:21.183437 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.183414 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-vcfnr"] Jun 09 11:30:21.650685 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.650654 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" path="/var/lib/kubelet/pods/d13561b8-bbb2-42d6-b2ea-495bd1774deb/volumes" Jun 09 11:30:21.863744 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.863720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" event={"ID":"d511c575-6885-404b-adef-8f06227258bd","Type":"ContainerDied","Data":"a538f99c0d3447f2af46ec91186a6e79bcb4fb4db9372cfc07c30221f2886eec"} Jun 09 11:30:21.864115 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.863731 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56" Jun 09 11:30:21.864115 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.863762 2560 scope.go:117] "RemoveContainer" containerID="6436d0556ccea73860c0f46a5c88134933ab073c818d1772b140aa7c211b1c0a" Jun 09 11:30:21.872194 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.872178 2560 scope.go:117] "RemoveContainer" containerID="67afbf92059f8d573b7914d777d0d9f496ccf08177ee6efc6fd826de0acb6b1c" Jun 09 11:30:21.879405 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.879392 2560 scope.go:117] "RemoveContainer" containerID="a89098d65294973e7e01923ca69e4c1eaaf8f8b42dc5b95c89a4df0b2fd74fa0" Jun 09 11:30:21.883827 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.883808 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:30:21.886923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.886902 2560 scope.go:117] "RemoveContainer" containerID="7b8885b245c2f368f6dcee2dfa5e3472b2ee706beae7f6997483f263718e5102" Jun 09 11:30:21.889717 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.889699 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-wrb56"] Jun 09 11:30:21.893758 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.893742 2560 scope.go:117] "RemoveContainer" containerID="a9fff8ee687e580ee1ac1f06ec8564a3ef2f0593d984fdc6263b61773be214f8" Jun 09 11:30:21.900219 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:21.900203 2560 scope.go:117] "RemoveContainer" containerID="5cbaeca55cf799d1ddcaabf0160dbe2d12969ef3ada8deed059b6f0e3d2594f0" Jun 09 11:30:23.650184 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:23.650153 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d511c575-6885-404b-adef-8f06227258bd" path="/var/lib/kubelet/pods/d511c575-6885-404b-adef-8f06227258bd/volumes" Jun 09 11:30:39.012558 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.012527 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:30:39.012959 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.012735 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" podUID="df719a2d-f019-4fad-8ccd-f92a17befebf" containerName="manager" containerID="cri-o://229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0" gracePeriod=10 Jun 09 11:30:39.240512 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.240490 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:30:39.393294 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.393261 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbjvg\" (UniqueName: \"kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg\") pod \"df719a2d-f019-4fad-8ccd-f92a17befebf\" (UID: \"df719a2d-f019-4fad-8ccd-f92a17befebf\") " Jun 09 11:30:39.395230 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.395202 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg" (OuterVolumeSpecName: "kube-api-access-zbjvg") pod "df719a2d-f019-4fad-8ccd-f92a17befebf" (UID: "df719a2d-f019-4fad-8ccd-f92a17befebf"). InnerVolumeSpecName "kube-api-access-zbjvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:30:39.494159 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.494134 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbjvg\" (UniqueName: \"kubernetes.io/projected/df719a2d-f019-4fad-8ccd-f92a17befebf-kube-api-access-zbjvg\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:30:39.933400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.933319 2560 generic.go:356] "Generic (PLEG): container finished" podID="df719a2d-f019-4fad-8ccd-f92a17befebf" containerID="229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0" exitCode=0 Jun 09 11:30:39.933400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.933365 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" Jun 09 11:30:39.933400 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.933378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" event={"ID":"df719a2d-f019-4fad-8ccd-f92a17befebf","Type":"ContainerDied","Data":"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0"} Jun 09 11:30:39.933636 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.933417 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7" event={"ID":"df719a2d-f019-4fad-8ccd-f92a17befebf","Type":"ContainerDied","Data":"3682bb194110b86a494fac1e6709b4dd500c0c20ec9a47daeea9cdef4199e88c"} Jun 09 11:30:39.933636 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.933439 2560 scope.go:117] "RemoveContainer" containerID="229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0" Jun 09 11:30:39.941024 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.940985 2560 scope.go:117] "RemoveContainer" containerID="229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0" Jun 09 11:30:39.941249 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:30:39.941231 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0\": container with ID starting with 229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0 not found: ID does not exist" containerID="229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0" Jun 09 11:30:39.941297 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.941257 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0"} err="failed to get container status \"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0\": rpc error: code = NotFound desc = could not find container \"229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0\": container with ID starting with 229bf38dbde9dd1f416f3ce8d41b12b888f6965ba8797d80935899db5b0ad0d0 not found: ID does not exist" Jun 09 11:30:39.949698 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.949677 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:30:39.960712 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:39.960692 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-sk7q7"] Jun 09 11:30:41.652121 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:41.652094 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df719a2d-f019-4fad-8ccd-f92a17befebf" path="/var/lib/kubelet/pods/df719a2d-f019-4fad-8ccd-f92a17befebf/volumes" Jun 09 11:30:43.496427 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496400 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7"] Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496685 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496697 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496705 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496711 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496722 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df719a2d-f019-4fad-8ccd-f92a17befebf" containerName="manager" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496727 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="df719a2d-f019-4fad-8ccd-f92a17befebf" containerName="manager" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496736 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="feast-init" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496741 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="feast-init" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496748 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="registry" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496753 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="registry" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496759 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496765 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496771 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496776 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496783 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="feast-apply" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496788 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="feast-apply" Jun 09 11:30:43.496787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496793 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="feast-init" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496798 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="feast-init" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496807 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496812 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496816 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496820 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496825 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="feast-apply" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496830 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="feast-apply" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496869 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496877 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="registry" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496883 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="ui" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496888 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="offline" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496894 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="df719a2d-f019-4fad-8ccd-f92a17befebf" containerName="manager" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496899 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="ui" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496905 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d13561b8-bbb2-42d6-b2ea-495bd1774deb" containerName="online" Jun 09 11:30:43.497270 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.496909 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d511c575-6885-404b-adef-8f06227258bd" containerName="online" Jun 09 11:30:43.500804 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.500788 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:43.503284 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.503266 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jun 09 11:30:43.504261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.504231 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jun 09 11:30:43.504326 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.504244 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-wtxb9\"" Jun 09 11:30:43.507418 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.507287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7"] Jun 09 11:30:43.624649 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.624620 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkhwp\" (UniqueName: \"kubernetes.io/projected/cb68f680-d617-408b-8312-f8fd5d5e3e8e-kube-api-access-dkhwp\") pod \"feast-operator-controller-manager-6d54c7db9d-dgbv7\" (UID: \"cb68f680-d617-408b-8312-f8fd5d5e3e8e\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:43.725383 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.725346 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkhwp\" (UniqueName: \"kubernetes.io/projected/cb68f680-d617-408b-8312-f8fd5d5e3e8e-kube-api-access-dkhwp\") pod \"feast-operator-controller-manager-6d54c7db9d-dgbv7\" (UID: \"cb68f680-d617-408b-8312-f8fd5d5e3e8e\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:43.734437 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.734415 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkhwp\" (UniqueName: \"kubernetes.io/projected/cb68f680-d617-408b-8312-f8fd5d5e3e8e-kube-api-access-dkhwp\") pod \"feast-operator-controller-manager-6d54c7db9d-dgbv7\" (UID: \"cb68f680-d617-408b-8312-f8fd5d5e3e8e\") " pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:43.811343 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.811322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:43.934356 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.934242 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7"] Jun 09 11:30:43.936843 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:30:43.936813 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb68f680_d617_408b_8312_f8fd5d5e3e8e.slice/crio-dfc083ba03aaf591e6d1efd320760062bd3064e5af257284bc0b9fffa2b74638 WatchSource:0}: Error finding container dfc083ba03aaf591e6d1efd320760062bd3064e5af257284bc0b9fffa2b74638: Status 404 returned error can't find the container with id dfc083ba03aaf591e6d1efd320760062bd3064e5af257284bc0b9fffa2b74638 Jun 09 11:30:43.949298 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:43.949261 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" event={"ID":"cb68f680-d617-408b-8312-f8fd5d5e3e8e","Type":"ContainerStarted","Data":"dfc083ba03aaf591e6d1efd320760062bd3064e5af257284bc0b9fffa2b74638"} Jun 09 11:30:44.954774 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:44.954735 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" event={"ID":"cb68f680-d617-408b-8312-f8fd5d5e3e8e","Type":"ContainerStarted","Data":"1d1920ea3758263ddcf6b315cc67703bc278ba91d6c1281929133e6074ab2985"} Jun 09 11:30:44.955218 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:44.954863 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:30:44.973310 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:44.973262 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" podStartSLOduration=1.9732492640000001 podStartE2EDuration="1.973249264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:30:44.972430057 +0000 UTC m=+1383.789119863" watchObservedRunningTime="2026-06-09 11:30:44.973249264 +0000 UTC m=+1383.789939068" Jun 09 11:30:55.959987 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:30:55.959956 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-6d54c7db9d-dgbv7" Jun 09 11:31:00.450014 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.449981 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:31:00.455190 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.455172 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.457935 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.457916 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 09 11:31:00.458049 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.457921 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-fg7gv\"" Jun 09 11:31:00.458049 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.457960 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 09 11:31:00.459149 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.459122 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 09 11:31:00.459149 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.459137 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 09 11:31:00.459149 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.459144 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 09 11:31:00.459330 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.459218 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 09 11:31:00.466055 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.466035 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:31:00.541387 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541361 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.541387 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vt5w\" (UniqueName: \"kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.541643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.541643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.541643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541554 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.541643 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.541616 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642651 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vt5w\" (UniqueName: \"kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.642787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.642765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.643203 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.643173 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.645112 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.645093 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.645233 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.645214 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.645274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.645226 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.645312 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.645280 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.650795 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.650775 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vt5w\" (UniqueName: \"kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w\") pod \"feast-simple-feast-setup-75c64c888b-xv9gm\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.777851 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.777769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:00.906364 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:00.906340 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:31:00.908528 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:31:00.908504 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89cc62cf_c9e9_4850_ae27_50aa4f9c4fbe.slice/crio-0d2c959a1ec8acf2a44fccdfeff19149de409c01f466ad2ef96bb6e4469a1620 WatchSource:0}: Error finding container 0d2c959a1ec8acf2a44fccdfeff19149de409c01f466ad2ef96bb6e4469a1620: Status 404 returned error can't find the container with id 0d2c959a1ec8acf2a44fccdfeff19149de409c01f466ad2ef96bb6e4469a1620 Jun 09 11:31:01.013897 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:01.013866 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"bae645d95c065b6527097eff24562d89e10a1731401e3a261c0d190c7430c974"} Jun 09 11:31:01.014081 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:01.013908 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"0d2c959a1ec8acf2a44fccdfeff19149de409c01f466ad2ef96bb6e4469a1620"} Jun 09 11:31:06.032652 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:06.032618 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="bae645d95c065b6527097eff24562d89e10a1731401e3a261c0d190c7430c974" exitCode=0 Jun 09 11:31:06.032997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:06.032690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"bae645d95c065b6527097eff24562d89e10a1731401e3a261c0d190c7430c974"} Jun 09 11:31:07.037692 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:07.037661 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"825f7fa522962412b5cd4a38fef503fe76f681fdbafdd50488fa0bc741c88d28"} Jun 09 11:31:12.057706 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:12.057674 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="825f7fa522962412b5cd4a38fef503fe76f681fdbafdd50488fa0bc741c88d28" exitCode=0 Jun 09 11:31:12.058059 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:12.057748 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"825f7fa522962412b5cd4a38fef503fe76f681fdbafdd50488fa0bc741c88d28"} Jun 09 11:31:13.082180 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:13.082135 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"af08bdcba9043cf2af8da28aae1bc2838590d14dec84c919419a710277c6475f"} Jun 09 11:31:13.082632 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:13.082203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"27caf7eb60c3f9c763b6d1bd65f5d87477ef72b3e4f48a7b25f4d88030a70cce"} Jun 09 11:31:13.082632 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:13.082223 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"5aa6bc8c9c2afa326473b19e237610e768d2e6c736cd3e474e4ff3b442dafcbe"} Jun 09 11:31:13.082632 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:13.082241 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerStarted","Data":"b55a3e9fbae438068591ef422660b55d68ec45fb58293e9eb3659fe88710a1c1"} Jun 09 11:31:13.108072 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:13.108004 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podStartSLOduration=13.107983417 podStartE2EDuration="13.107983417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:31:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:31:13.106497226 +0000 UTC m=+1411.923187044" watchObservedRunningTime="2026-06-09 11:31:13.107983417 +0000 UTC m=+1411.924673224" Jun 09 11:31:15.778656 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.778624 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:15.779183 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.779154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:15.779183 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.779190 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:15.779381 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.779209 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:15.780631 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.780601 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 09 11:31:15.780833 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.780613 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.22:8816: connect: connection refused" Jun 09 11:31:15.781006 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.780626 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.22:6571: connect: connection refused" Jun 09 11:31:15.781109 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:15.780664 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 09 11:31:18.779312 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779284 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.779680 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779416 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.779732 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779694 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.779820 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779800 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.779885 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779845 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.779885 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.779865 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.780141 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.780116 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.780271 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.780231 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.780331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.780282 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:18.783551 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:18.783535 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:19.111545 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:19.111514 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:31:19.114378 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:31:19.114359 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:32:21.343826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.343794 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:32:21.344318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.344076 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="registry" containerID="cri-o://b55a3e9fbae438068591ef422660b55d68ec45fb58293e9eb3659fe88710a1c1" gracePeriod=30 Jun 09 11:32:21.344318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.344114 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" containerID="cri-o://af08bdcba9043cf2af8da28aae1bc2838590d14dec84c919419a710277c6475f" gracePeriod=30 Jun 09 11:32:21.344318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.344131 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" containerID="cri-o://5aa6bc8c9c2afa326473b19e237610e768d2e6c736cd3e474e4ff3b442dafcbe" gracePeriod=30 Jun 09 11:32:21.344318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.344165 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="offline" containerID="cri-o://27caf7eb60c3f9c763b6d1bd65f5d87477ef72b3e4f48a7b25f4d88030a70cce" gracePeriod=30 Jun 09 11:32:21.652724 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.652648 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:32:21.656596 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.656575 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.659053 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.659023 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-4jpqn\"" Jun 09 11:32:21.671545 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.671518 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:32:21.759316 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.759507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759362 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6fcl\" (UniqueName: \"kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.759507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759438 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.759507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759503 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.759682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.759682 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.759664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860404 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6fcl\" (UniqueName: \"kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860482 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860530 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860602 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860663 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860618 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860663 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:32:21.860619 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jun 09 11:32:21.860760 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.860813 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:32:21.860782 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls podName:f678a4f9-ee34-448d-b836-8729cd1d59ae nodeName:}" failed. No retries permitted until 2026-06-09 11:32:22.360760828 +0000 UTC m=+1481.177450615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls") pod "feast-simple-feast-setup-75c64c888b-gt7b8" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae") : secret "feast-simple-feast-setup-offline-tls" not found Jun 09 11:32:21.860975 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.860940 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.862785 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.862759 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.863249 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.863218 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.863376 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.863353 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:21.868887 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:21.868863 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6fcl\" (UniqueName: \"kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:22.357817 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.357789 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="af08bdcba9043cf2af8da28aae1bc2838590d14dec84c919419a710277c6475f" exitCode=0 Jun 09 11:32:22.357817 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.357810 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="5aa6bc8c9c2afa326473b19e237610e768d2e6c736cd3e474e4ff3b442dafcbe" exitCode=0 Jun 09 11:32:22.358231 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.357871 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"af08bdcba9043cf2af8da28aae1bc2838590d14dec84c919419a710277c6475f"} Jun 09 11:32:22.358231 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.357913 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"5aa6bc8c9c2afa326473b19e237610e768d2e6c736cd3e474e4ff3b442dafcbe"} Jun 09 11:32:22.365190 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.365169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:22.367470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.367444 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") pod \"feast-simple-feast-setup-75c64c888b-gt7b8\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:22.566957 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.566924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:22.693032 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:22.692999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:32:22.693588 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:32:22.693545 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf678a4f9_ee34_448d_b836_8729cd1d59ae.slice/crio-27229c246b7187095ef3a804fdc8937668172db3ec2fe7320e7e8066dd49b0d3 WatchSource:0}: Error finding container 27229c246b7187095ef3a804fdc8937668172db3ec2fe7320e7e8066dd49b0d3: Status 404 returned error can't find the container with id 27229c246b7187095ef3a804fdc8937668172db3ec2fe7320e7e8066dd49b0d3 Jun 09 11:32:23.363907 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:23.363870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434"} Jun 09 11:32:23.364272 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:23.363921 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"27229c246b7187095ef3a804fdc8937668172db3ec2fe7320e7e8066dd49b0d3"} Jun 09 11:32:27.384677 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:27.384643 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434" exitCode=0 Jun 09 11:32:27.385085 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:27.384720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434"} Jun 09 11:32:28.390780 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:28.390747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68"} Jun 09 11:32:28.780420 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:28.780324 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 09 11:32:29.111963 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:29.111929 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 09 11:32:33.412666 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:33.412636 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68" exitCode=0 Jun 09 11:32:33.413048 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:33.412710 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68"} Jun 09 11:32:34.422981 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.422938 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97"} Jun 09 11:32:34.423458 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.423002 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2"} Jun 09 11:32:34.423458 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.423024 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5"} Jun 09 11:32:34.423458 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.423044 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerStarted","Data":"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e"} Jun 09 11:32:34.449512 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.449434 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podStartSLOduration=13.449412597 podStartE2EDuration="13.449412597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:32:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:32:34.446104721 +0000 UTC m=+1493.262794527" watchObservedRunningTime="2026-06-09 11:32:34.449412597 +0000 UTC m=+1493.266102405" Jun 09 11:32:34.568191 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.568013 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:34.568191 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.568092 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:34.568191 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.568112 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:34.568191 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.568144 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:34.570405 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.570361 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Jun 09 11:32:34.570537 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.570401 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 09 11:32:34.570537 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.570406 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 09 11:32:34.570537 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:34.570464 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Jun 09 11:32:37.568178 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:37.568135 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.23:8816: connect: connection refused" Jun 09 11:32:37.568826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:37.568128 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.23:6571: connect: connection refused" Jun 09 11:32:37.568966 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:37.568147 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 09 11:32:37.568966 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:37.568169 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 09 11:32:38.780973 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:38.780924 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 09 11:32:39.111880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:39.111848 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 09 11:32:40.567755 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.567726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568208 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.567867 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568208 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568115 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568376 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568358 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568505 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568421 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568505 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568441 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568703 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568684 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568768 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568755 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.568825 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.568813 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:40.572244 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:40.572223 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:41.458474 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:41.458448 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:41.461289 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:41.461264 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:32:41.700261 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:41.700235 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:32:41.701266 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:41.701244 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:32:48.779963 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:48.779922 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.22:8443: connect: connection refused" Jun 09 11:32:48.780493 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:48.780051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:32:49.111815 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:49.111777 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" probeResult="failure" output="Get \"https://10.134.0.22:6567/health\": dial tcp 10.134.0.22:6567: connect: connection refused" Jun 09 11:32:49.111976 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:49.111908 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:32:51.499487 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:51.499463 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="27caf7eb60c3f9c763b6d1bd65f5d87477ef72b3e4f48a7b25f4d88030a70cce" exitCode=137 Jun 09 11:32:51.499487 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:51.499483 2560 generic.go:356] "Generic (PLEG): container finished" podID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerID="b55a3e9fbae438068591ef422660b55d68ec45fb58293e9eb3659fe88710a1c1" exitCode=137 Jun 09 11:32:51.499821 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:51.499511 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"27caf7eb60c3f9c763b6d1bd65f5d87477ef72b3e4f48a7b25f4d88030a70cce"} Jun 09 11:32:51.499821 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:51.499540 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"b55a3e9fbae438068591ef422660b55d68ec45fb58293e9eb3659fe88710a1c1"} Jun 09 11:32:51.988677 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:51.988649 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:32:52.014274 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vt5w\" (UniqueName: \"kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014385 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014289 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014385 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014365 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014500 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014396 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014500 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014432 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014500 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014456 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls\") pod \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\" (UID: \"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe\") " Jun 09 11:32:52.014992 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.014962 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data" (OuterVolumeSpecName: "feast-data") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:32:52.016501 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.016467 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:32:52.016629 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.016544 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls" (OuterVolumeSpecName: "online-tls") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:32:52.016629 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.016615 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:32:52.016725 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.016646 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:32:52.016725 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.016705 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w" (OuterVolumeSpecName: "kube-api-access-5vt5w") pod "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" (UID: "89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe"). InnerVolumeSpecName "kube-api-access-5vt5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:32:52.115555 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115499 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vt5w\" (UniqueName: \"kubernetes.io/projected/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-kube-api-access-5vt5w\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.115555 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115523 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.115555 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115533 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.115555 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115541 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.115555 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115549 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.115787 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.115591 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:32:52.506331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.506260 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" Jun 09 11:32:52.506331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.506263 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm" event={"ID":"89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe","Type":"ContainerDied","Data":"0d2c959a1ec8acf2a44fccdfeff19149de409c01f466ad2ef96bb6e4469a1620"} Jun 09 11:32:52.506331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.506310 2560 scope.go:117] "RemoveContainer" containerID="af08bdcba9043cf2af8da28aae1bc2838590d14dec84c919419a710277c6475f" Jun 09 11:32:52.514914 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.514895 2560 scope.go:117] "RemoveContainer" containerID="27caf7eb60c3f9c763b6d1bd65f5d87477ef72b3e4f48a7b25f4d88030a70cce" Jun 09 11:32:52.521911 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.521893 2560 scope.go:117] "RemoveContainer" containerID="5aa6bc8c9c2afa326473b19e237610e768d2e6c736cd3e474e4ff3b442dafcbe" Jun 09 11:32:52.529674 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.529644 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:32:52.529793 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.529738 2560 scope.go:117] "RemoveContainer" containerID="b55a3e9fbae438068591ef422660b55d68ec45fb58293e9eb3659fe88710a1c1" Jun 09 11:32:52.531930 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.531912 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-xv9gm"] Jun 09 11:32:52.537175 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.537159 2560 scope.go:117] "RemoveContainer" containerID="825f7fa522962412b5cd4a38fef503fe76f681fdbafdd50488fa0bc741c88d28" Jun 09 11:32:52.545761 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:52.545745 2560 scope.go:117] "RemoveContainer" containerID="bae645d95c065b6527097eff24562d89e10a1731401e3a261c0d190c7430c974" Jun 09 11:32:53.651067 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:32:53.651032 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" path="/var/lib/kubelet/pods/89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe/volumes" Jun 09 11:33:43.004672 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.004638 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005551 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005595 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005616 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="feast-apply" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005625 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="feast-apply" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005657 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="feast-init" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005665 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="feast-init" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005688 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="registry" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005697 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="registry" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005716 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005725 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005734 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="offline" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005743 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="offline" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005915 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="ui" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005926 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="registry" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005943 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="online" Jun 09 11:33:43.008937 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.005953 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="89cc62cf-c9e9-4850-ae27-50aa4f9c4fbe" containerName="offline" Jun 09 11:33:43.011209 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.011186 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.013785 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.013766 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 09 11:33:43.014056 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014032 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rjlzf\"" Jun 09 11:33:43.014164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014080 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 09 11:33:43.014164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014107 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 09 11:33:43.014164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014083 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 09 11:33:43.014164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014080 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 09 11:33:43.014368 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.014223 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 09 11:33:43.019248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.019218 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:33:43.177834 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.177800 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd8tf\" (UniqueName: \"kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.177995 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.177839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.177995 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.177857 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.177995 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.177876 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.177995 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.177976 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.178146 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.178004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.278867 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.278800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.278867 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.278837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279021 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:33:43.278932 2560 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jun 09 11:33:43.279021 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.278957 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd8tf\" (UniqueName: \"kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279021 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:33:43.279002 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls podName:0e245c9d-335e-4403-8a21-1e802e8e6f07 nodeName:}" failed. No retries permitted until 2026-06-09 11:33:43.778982774 +0000 UTC m=+1562.595672560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-x8d52" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07") : secret "feast-simple-feast-remote-setup-online-tls" not found Jun 09 11:33:43.279151 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.279021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279151 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.279038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279151 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.279062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279310 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:33:43.279164 2560 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 09 11:33:43.279310 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:33:43.279201 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls podName:0e245c9d-335e-4403-8a21-1e802e8e6f07 nodeName:}" failed. No retries permitted until 2026-06-09 11:33:43.779190756 +0000 UTC m=+1562.595880539 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls") pod "feast-simple-feast-remote-setup-669bcf45fc-x8d52" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jun 09 11:33:43.279482 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.279451 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.279614 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.279506 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.281318 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.281303 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.289452 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.289433 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd8tf\" (UniqueName: \"kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.783849 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.783821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.783989 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.783882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.786262 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.786239 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.786331 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.786239 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") pod \"feast-simple-feast-remote-setup-669bcf45fc-x8d52\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:43.920296 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:43.920266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:44.046710 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:44.046672 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:33:44.048491 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:33:44.048465 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e245c9d_335e_4403_8a21_1e802e8e6f07.slice/crio-7762ae4335c9dda1959124f74a75e5e7c36feb1f63434cb4a83cd2f5f2091489 WatchSource:0}: Error finding container 7762ae4335c9dda1959124f74a75e5e7c36feb1f63434cb4a83cd2f5f2091489: Status 404 returned error can't find the container with id 7762ae4335c9dda1959124f74a75e5e7c36feb1f63434cb4a83cd2f5f2091489 Jun 09 11:33:44.707142 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:44.707102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"69ec863c3b004593bcbaaad6d94350f5ce46737c095976c8e077a0be38ddd95a"} Jun 09 11:33:44.707329 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:44.707158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"7762ae4335c9dda1959124f74a75e5e7c36feb1f63434cb4a83cd2f5f2091489"} Jun 09 11:33:48.722328 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:48.722256 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerID="69ec863c3b004593bcbaaad6d94350f5ce46737c095976c8e077a0be38ddd95a" exitCode=0 Jun 09 11:33:48.722759 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:48.722325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"69ec863c3b004593bcbaaad6d94350f5ce46737c095976c8e077a0be38ddd95a"} Jun 09 11:33:49.728016 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:49.727982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"ef954dae2e95be75e7a538973c1183bd8fbfc8476a0fdc20fc4016d058670c3c"} Jun 09 11:33:54.749791 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:54.749754 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerID="ef954dae2e95be75e7a538973c1183bd8fbfc8476a0fdc20fc4016d058670c3c" exitCode=0 Jun 09 11:33:54.750164 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:54.749808 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"ef954dae2e95be75e7a538973c1183bd8fbfc8476a0fdc20fc4016d058670c3c"} Jun 09 11:33:55.758428 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.758387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"5a240817af1c6ca92baa50431cd701974e89ac431ea62c611bc853a6ce409390"} Jun 09 11:33:55.758932 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.758443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"847f1f48b4ab4f55eef80da56ec8169150ad8b639057b48069cad0d9e1bc2eaf"} Jun 09 11:33:55.758932 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.758464 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerStarted","Data":"e421383a48385f1c9f35b431c75fe10e9f462b2d81c09a5775abf893b803351b"} Jun 09 11:33:55.784336 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.784293 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podStartSLOduration=13.784276584 podStartE2EDuration="13.784276584s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 11:33:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:33:55.781437354 +0000 UTC m=+1574.598127159" watchObservedRunningTime="2026-06-09 11:33:55.784276584 +0000 UTC m=+1574.600966388" Jun 09 11:33:55.920829 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.920792 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:55.921032 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.920853 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:55.921032 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.920876 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:33:55.922523 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.922490 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 11:33:55.922669 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.922550 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jun 09 11:33:55.922669 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:55.922490 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Jun 09 11:33:58.921511 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:58.921467 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Jun 09 11:33:58.921892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:58.921547 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jun 09 11:33:58.921892 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:33:58.921475 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 11:34:01.921288 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.921245 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.921793 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.921464 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.921793 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.921702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.921793 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.921730 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.921969 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.921953 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.922023 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.922013 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:01.925352 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:01.925336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:02.791833 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:02.791805 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:34:02.794510 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:34:02.794488 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:35:04.095401 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.095367 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:35:04.097802 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.095655 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" containerID="cri-o://e421383a48385f1c9f35b431c75fe10e9f462b2d81c09a5775abf893b803351b" gracePeriod=30 Jun 09 11:35:04.097802 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.095729 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" containerID="cri-o://5a240817af1c6ca92baa50431cd701974e89ac431ea62c611bc853a6ce409390" gracePeriod=30 Jun 09 11:35:04.097802 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.095729 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" containerID="cri-o://847f1f48b4ab4f55eef80da56ec8169150ad8b639057b48069cad0d9e1bc2eaf" gracePeriod=30 Jun 09 11:35:04.240674 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.240641 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:35:04.241098 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.240937 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" containerID="cri-o://e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" gracePeriod=30 Jun 09 11:35:04.241098 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.240948 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" containerID="cri-o://1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" gracePeriod=30 Jun 09 11:35:04.241098 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.240959 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" containerID="cri-o://12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" gracePeriod=30 Jun 09 11:35:04.241351 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:04.240912 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" containerID="cri-o://6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" gracePeriod=30 Jun 09 11:35:05.037428 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.037357 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" exitCode=0 Jun 09 11:35:05.037428 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.037381 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" exitCode=0 Jun 09 11:35:05.037594 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.037431 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97"} Jun 09 11:35:05.037594 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.037471 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5"} Jun 09 11:35:05.039880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.039860 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerID="5a240817af1c6ca92baa50431cd701974e89ac431ea62c611bc853a6ce409390" exitCode=0 Jun 09 11:35:05.039880 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.039876 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerID="e421383a48385f1c9f35b431c75fe10e9f462b2d81c09a5775abf893b803351b" exitCode=0 Jun 09 11:35:05.039998 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.039919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"5a240817af1c6ca92baa50431cd701974e89ac431ea62c611bc853a6ce409390"} Jun 09 11:35:05.039998 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:05.039943 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"e421383a48385f1c9f35b431c75fe10e9f462b2d81c09a5775abf893b803351b"} Jun 09 11:35:10.568738 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:10.568673 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 09 11:35:11.458818 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:11.458776 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 09 11:35:11.922113 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:11.922080 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 11:35:12.792388 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:12.792352 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jun 09 11:35:20.568785 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:20.568732 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 09 11:35:21.459346 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:21.459312 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 09 11:35:21.921975 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:21.921938 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 11:35:22.792304 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:22.792268 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jun 09 11:35:30.569268 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:30.569223 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.23:8443: connect: connection refused" Jun 09 11:35:30.569717 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:30.569374 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:35:31.459091 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:31.459058 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" probeResult="failure" output="Get \"https://10.134.0.23:6567/health\": dial tcp 10.134.0.23:6567: connect: connection refused" Jun 09 11:35:31.459260 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:31.459174 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:35:31.922572 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:31.922529 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jun 09 11:35:31.922928 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:31.922668 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:35:32.791928 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:32.791885 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jun 09 11:35:32.792088 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:32.791993 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:35:34.152432 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.152404 2560 generic.go:356] "Generic (PLEG): container finished" podID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerID="847f1f48b4ab4f55eef80da56ec8169150ad8b639057b48069cad0d9e1bc2eaf" exitCode=137 Jun 09 11:35:34.152746 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.152481 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"847f1f48b4ab4f55eef80da56ec8169150ad8b639057b48069cad0d9e1bc2eaf"} Jun 09 11:35:34.729797 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.729778 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:35:34.829950 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.829927 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.829969 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830004 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830066 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830024 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd8tf\" (UniqueName: \"kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830195 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830088 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830195 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830114 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") pod \"0e245c9d-335e-4403-8a21-1e802e8e6f07\" (UID: \"0e245c9d-335e-4403-8a21-1e802e8e6f07\") " Jun 09 11:35:34.830365 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830337 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 11:35:34.830460 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.830416 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data" (OuterVolumeSpecName: "feast-data") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:35:34.832255 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.832228 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:34.832360 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.832252 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:34.832360 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.832266 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf" (OuterVolumeSpecName: "kube-api-access-dd8tf") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "kube-api-access-dd8tf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:35:34.832360 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.832276 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls" (OuterVolumeSpecName: "online-tls") pod "0e245c9d-335e-4403-8a21-1e802e8e6f07" (UID: "0e245c9d-335e-4403-8a21-1e802e8e6f07"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:34.876884 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.876857 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:35:34.931669 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931646 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0e245c9d-335e-4403-8a21-1e802e8e6f07-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:34.931669 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931671 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/0e245c9d-335e-4403-8a21-1e802e8e6f07-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:34.931823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931684 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:34.931823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931695 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd8tf\" (UniqueName: \"kubernetes.io/projected/0e245c9d-335e-4403-8a21-1e802e8e6f07-kube-api-access-dd8tf\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:34.931823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931707 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:34.931823 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:34.931742 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0e245c9d-335e-4403-8a21-1e802e8e6f07-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.032281 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032256 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032292 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032311 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032341 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6fcl\" (UniqueName: \"kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032387 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032470 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032440 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls\") pod \"f678a4f9-ee34-448d-b836-8729cd1d59ae\" (UID: \"f678a4f9-ee34-448d-b836-8729cd1d59ae\") " Jun 09 11:35:35.032851 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.032826 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data" (OuterVolumeSpecName: "feast-data") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 11:35:35.034421 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.034400 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:35.034501 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.034399 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl" (OuterVolumeSpecName: "kube-api-access-h6fcl") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "kube-api-access-h6fcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 11:35:35.034501 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.034420 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls" (OuterVolumeSpecName: "online-tls") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:35.034707 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.034686 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:35.034707 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.034699 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f678a4f9-ee34-448d-b836-8729cd1d59ae" (UID: "f678a4f9-ee34-448d-b836-8729cd1d59ae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133349 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-offline-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133380 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-ui-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133390 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6fcl\" (UniqueName: \"kubernetes.io/projected/f678a4f9-ee34-448d-b836-8729cd1d59ae-kube-api-access-h6fcl\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133399 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-registry-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133407 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f678a4f9-ee34-448d-b836-8729cd1d59ae-online-tls\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.133406 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.133414 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f678a4f9-ee34-448d-b836-8729cd1d59ae-feast-data\") on node \"ip-10-0-130-8.ec2.internal\" DevicePath \"\"" Jun 09 11:35:35.159460 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.159442 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" Jun 09 11:35:35.159804 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.159475 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52" event={"ID":"0e245c9d-335e-4403-8a21-1e802e8e6f07","Type":"ContainerDied","Data":"7762ae4335c9dda1959124f74a75e5e7c36feb1f63434cb4a83cd2f5f2091489"} Jun 09 11:35:35.159804 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.159536 2560 scope.go:117] "RemoveContainer" containerID="5a240817af1c6ca92baa50431cd701974e89ac431ea62c611bc853a6ce409390" Jun 09 11:35:35.162541 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162521 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" exitCode=137 Jun 09 11:35:35.162653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162542 2560 generic.go:356] "Generic (PLEG): container finished" podID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerID="6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" exitCode=137 Jun 09 11:35:35.162653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162579 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2"} Jun 09 11:35:35.162653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e"} Jun 09 11:35:35.162653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162633 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" event={"ID":"f678a4f9-ee34-448d-b836-8729cd1d59ae","Type":"ContainerDied","Data":"27229c246b7187095ef3a804fdc8937668172db3ec2fe7320e7e8066dd49b0d3"} Jun 09 11:35:35.162653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.162643 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8" Jun 09 11:35:35.167785 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.167729 2560 scope.go:117] "RemoveContainer" containerID="847f1f48b4ab4f55eef80da56ec8169150ad8b639057b48069cad0d9e1bc2eaf" Jun 09 11:35:35.175214 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.175200 2560 scope.go:117] "RemoveContainer" containerID="e421383a48385f1c9f35b431c75fe10e9f462b2d81c09a5775abf893b803351b" Jun 09 11:35:35.181530 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.181514 2560 scope.go:117] "RemoveContainer" containerID="ef954dae2e95be75e7a538973c1183bd8fbfc8476a0fdc20fc4016d058670c3c" Jun 09 11:35:35.188015 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.187994 2560 scope.go:117] "RemoveContainer" containerID="69ec863c3b004593bcbaaad6d94350f5ce46737c095976c8e077a0be38ddd95a" Jun 09 11:35:35.190301 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.190278 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:35:35.194476 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.194456 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-669bcf45fc-x8d52"] Jun 09 11:35:35.196652 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.196632 2560 scope.go:117] "RemoveContainer" containerID="e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" Jun 09 11:35:35.203503 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.203488 2560 scope.go:117] "RemoveContainer" containerID="1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" Jun 09 11:35:35.209852 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.209833 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:35:35.210028 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.210015 2560 scope.go:117] "RemoveContainer" containerID="12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" Jun 09 11:35:35.213891 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.213851 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-75c64c888b-gt7b8"] Jun 09 11:35:35.217205 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.217190 2560 scope.go:117] "RemoveContainer" containerID="6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" Jun 09 11:35:35.223543 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.223528 2560 scope.go:117] "RemoveContainer" containerID="e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68" Jun 09 11:35:35.229692 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.229677 2560 scope.go:117] "RemoveContainer" containerID="be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434" Jun 09 11:35:35.235826 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.235812 2560 scope.go:117] "RemoveContainer" containerID="e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" Jun 09 11:35:35.236040 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.236025 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97\": container with ID starting with e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97 not found: ID does not exist" containerID="e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" Jun 09 11:35:35.236081 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236047 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97"} err="failed to get container status \"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97\": rpc error: code = NotFound desc = could not find container \"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97\": container with ID starting with e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97 not found: ID does not exist" Jun 09 11:35:35.236081 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236063 2560 scope.go:117] "RemoveContainer" containerID="1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" Jun 09 11:35:35.236267 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.236253 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2\": container with ID starting with 1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2 not found: ID does not exist" containerID="1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" Jun 09 11:35:35.236308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236269 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2"} err="failed to get container status \"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2\": rpc error: code = NotFound desc = could not find container \"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2\": container with ID starting with 1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2 not found: ID does not exist" Jun 09 11:35:35.236308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236281 2560 scope.go:117] "RemoveContainer" containerID="12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" Jun 09 11:35:35.236471 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.236458 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5\": container with ID starting with 12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5 not found: ID does not exist" containerID="12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" Jun 09 11:35:35.236508 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236473 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5"} err="failed to get container status \"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5\": rpc error: code = NotFound desc = could not find container \"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5\": container with ID starting with 12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5 not found: ID does not exist" Jun 09 11:35:35.236508 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236484 2560 scope.go:117] "RemoveContainer" containerID="6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" Jun 09 11:35:35.236703 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.236687 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e\": container with ID starting with 6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e not found: ID does not exist" containerID="6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" Jun 09 11:35:35.236754 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236708 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e"} err="failed to get container status \"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e\": rpc error: code = NotFound desc = could not find container \"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e\": container with ID starting with 6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e not found: ID does not exist" Jun 09 11:35:35.236754 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236723 2560 scope.go:117] "RemoveContainer" containerID="e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68" Jun 09 11:35:35.236956 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.236938 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68\": container with ID starting with e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68 not found: ID does not exist" containerID="e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68" Jun 09 11:35:35.236997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236969 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68"} err="failed to get container status \"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68\": rpc error: code = NotFound desc = could not find container \"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68\": container with ID starting with e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68 not found: ID does not exist" Jun 09 11:35:35.236997 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.236984 2560 scope.go:117] "RemoveContainer" containerID="be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434" Jun 09 11:35:35.237186 ip-10-0-130-8 kubenswrapper[2560]: E0609 11:35:35.237171 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434\": container with ID starting with be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434 not found: ID does not exist" containerID="be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434" Jun 09 11:35:35.237228 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237189 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434"} err="failed to get container status \"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434\": rpc error: code = NotFound desc = could not find container \"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434\": container with ID starting with be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434 not found: ID does not exist" Jun 09 11:35:35.237228 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237201 2560 scope.go:117] "RemoveContainer" containerID="e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97" Jun 09 11:35:35.237397 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237382 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97"} err="failed to get container status \"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97\": rpc error: code = NotFound desc = could not find container \"e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97\": container with ID starting with e2b57a74d0b4312454b01ee6fc9c74178c9731a16220c7355320e57058f51a97 not found: ID does not exist" Jun 09 11:35:35.237447 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237397 2560 scope.go:117] "RemoveContainer" containerID="1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2" Jun 09 11:35:35.237617 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237600 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2"} err="failed to get container status \"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2\": rpc error: code = NotFound desc = could not find container \"1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2\": container with ID starting with 1736f1fbb8716f34d336fd3ac96a092329dd651139317f0cad0be9709a4a09c2 not found: ID does not exist" Jun 09 11:35:35.237617 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237617 2560 scope.go:117] "RemoveContainer" containerID="12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5" Jun 09 11:35:35.237820 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237794 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5"} err="failed to get container status \"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5\": rpc error: code = NotFound desc = could not find container \"12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5\": container with ID starting with 12b5fbbf4e961d443a8eba09a634705d2357d31ea236b4f08375f255656fd4c5 not found: ID does not exist" Jun 09 11:35:35.237820 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.237812 2560 scope.go:117] "RemoveContainer" containerID="6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e" Jun 09 11:35:35.238023 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.238005 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e"} err="failed to get container status \"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e\": rpc error: code = NotFound desc = could not find container \"6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e\": container with ID starting with 6edc458481fbe7888f0b68c4073d72beeb73cbe913c3a9c82214a3661434315e not found: ID does not exist" Jun 09 11:35:35.238070 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.238024 2560 scope.go:117] "RemoveContainer" containerID="e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68" Jun 09 11:35:35.238206 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.238192 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68"} err="failed to get container status \"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68\": rpc error: code = NotFound desc = could not find container \"e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68\": container with ID starting with e81bc04e63dfb3b6c1b231c8a08649fe17497976ae814b4816966a95e04a5d68 not found: ID does not exist" Jun 09 11:35:35.238249 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.238206 2560 scope.go:117] "RemoveContainer" containerID="be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434" Jun 09 11:35:35.238361 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.238344 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434"} err="failed to get container status \"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434\": rpc error: code = NotFound desc = could not find container \"be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434\": container with ID starting with be63ea4d21f0e2d5b2c55372c1abd8715bacaa2a91ec95b414ebea671e841434 not found: ID does not exist" Jun 09 11:35:35.650799 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.650775 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" path="/var/lib/kubelet/pods/0e245c9d-335e-4403-8a21-1e802e8e6f07/volumes" Jun 09 11:35:35.651286 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:35:35.651272 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" path="/var/lib/kubelet/pods/f678a4f9-ee34-448d-b836-8729cd1d59ae/volumes" Jun 09 11:36:14.756317 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:14.756242 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d2lzb_a67614cc-102f-483b-858c-f180e0540da9/global-pull-secret-syncer/0.log" Jun 09 11:36:14.932993 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:14.932966 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zhcbk_ff9cd805-64d9-4f13-9360-8461bb56d5b2/konnectivity-agent/0.log" Jun 09 11:36:14.954195 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:14.954173 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-8.ec2.internal_79ba099ecd0a9bcdc399892bd5724baa/haproxy/0.log" Jun 09 11:36:18.733987 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.733960 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/alertmanager/0.log" Jun 09 11:36:18.758033 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.758000 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/config-reloader/0.log" Jun 09 11:36:18.784579 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.784547 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/kube-rbac-proxy-web/0.log" Jun 09 11:36:18.830179 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.830161 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/kube-rbac-proxy/0.log" Jun 09 11:36:18.860010 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.859994 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/kube-rbac-proxy-metric/0.log" Jun 09 11:36:18.890353 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.890335 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/prom-label-proxy/0.log" Jun 09 11:36:18.915974 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:18.915930 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2b57d2dc-6df7-416a-96e0-59476b49d6c9/init-config-reloader/0.log" Jun 09 11:36:19.082729 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.082711 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6ddfbcd75b-hch67_213c7c8e-26bc-4038-be15-ed1bcc4480ae/metrics-server/0.log" Jun 09 11:36:19.113078 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.113060 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-sxrd7_6fdd4b06-f90f-4858-8716-534477fe901d/monitoring-plugin/0.log" Jun 09 11:36:19.152453 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.152428 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8g78d_f376332f-bbb7-4e06-97a1-a7a0f10674a7/node-exporter/0.log" Jun 09 11:36:19.175690 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.175643 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8g78d_f376332f-bbb7-4e06-97a1-a7a0f10674a7/kube-rbac-proxy/0.log" Jun 09 11:36:19.200057 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.200043 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8g78d_f376332f-bbb7-4e06-97a1-a7a0f10674a7/init-textfile/0.log" Jun 09 11:36:19.488916 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.488887 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/prometheus/0.log" Jun 09 11:36:19.517724 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.517703 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/config-reloader/0.log" Jun 09 11:36:19.543034 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.543017 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/thanos-sidecar/0.log" Jun 09 11:36:19.568334 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.568315 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/kube-rbac-proxy-web/0.log" Jun 09 11:36:19.591934 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.591917 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/kube-rbac-proxy/0.log" Jun 09 11:36:19.621224 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.621204 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/kube-rbac-proxy-thanos/0.log" Jun 09 11:36:19.650549 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.650531 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_ef6e1b10-50c4-45d2-b53a-918ba01e65e7/init-config-reloader/0.log" Jun 09 11:36:19.689637 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.689591 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-9249x_3d143709-9689-4915-a318-709894827de4/prometheus-operator/0.log" Jun 09 11:36:19.718999 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.718983 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-9249x_3d143709-9689-4915-a318-709894827de4/kube-rbac-proxy/0.log" Jun 09 11:36:19.796163 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.796144 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd644d585-7gn4g_e266599b-4e54-4441-9ee8-50dfbf229f56/telemeter-client/0.log" Jun 09 11:36:19.842870 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.842851 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd644d585-7gn4g_e266599b-4e54-4441-9ee8-50dfbf229f56/reload/0.log" Jun 09 11:36:19.874690 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.874670 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-dd644d585-7gn4g_e266599b-4e54-4441-9ee8-50dfbf229f56/kube-rbac-proxy/0.log" Jun 09 11:36:19.913671 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.913652 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/thanos-query/0.log" Jun 09 11:36:19.940248 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.940199 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/kube-rbac-proxy-web/0.log" Jun 09 11:36:19.987650 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:19.987626 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/kube-rbac-proxy/0.log" Jun 09 11:36:20.012173 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:20.012157 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/prom-label-proxy/0.log" Jun 09 11:36:20.035523 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:20.035507 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/kube-rbac-proxy-rules/0.log" Jun 09 11:36:20.057989 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:20.057970 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-56b7cb4894-kzg88_641bc884-e723-49c9-82c5-79b27c994999/kube-rbac-proxy-metrics/0.log" Jun 09 11:36:22.249912 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.249882 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg"] Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250161 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="feast-apply" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250171 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="feast-apply" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250180 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250186 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250193 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250199 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250210 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250216 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250225 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="feast-init" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250231 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="feast-init" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250240 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="feast-apply" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250245 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="feast-apply" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250252 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="feast-init" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250257 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="feast-init" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250263 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" Jun 09 11:36:22.250263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250268 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250275 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250280 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250287 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250292 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250297 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250301 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250344 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="online" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250350 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="offline" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250356 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="online" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250362 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250368 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e245c9d-335e-4403-8a21-1e802e8e6f07" containerName="ui" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250373 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="offline" Jun 09 11:36:22.250723 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.250379 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="f678a4f9-ee34-448d-b836-8729cd1d59ae" containerName="registry" Jun 09 11:36:22.255653 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.254715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.258417 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.258397 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbd8c\"/\"openshift-service-ca.crt\"" Jun 09 11:36:22.258529 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.258462 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vbd8c\"/\"default-dockercfg-mmc2g\"" Jun 09 11:36:22.259534 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.259512 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vbd8c\"/\"kube-root-ca.crt\"" Jun 09 11:36:22.263873 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.263844 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg"] Jun 09 11:36:22.374685 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.374655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-lib-modules\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.374832 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.374697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-proc\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.374832 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.374729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-sys\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.374832 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.374753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-podres\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.374832 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.374785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd9gd\" (UniqueName: \"kubernetes.io/projected/8967d9da-1684-410a-a6e0-97b99e4f192c-kube-api-access-wd9gd\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475604 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd9gd\" (UniqueName: \"kubernetes.io/projected/8967d9da-1684-410a-a6e0-97b99e4f192c-kube-api-access-wd9gd\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475767 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-lib-modules\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475767 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-proc\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475767 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-sys\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475769 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-sys\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475774 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-proc\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-podres\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475824 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-lib-modules\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.475923 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.475884 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8967d9da-1684-410a-a6e0-97b99e4f192c-podres\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.484180 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.484155 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd9gd\" (UniqueName: \"kubernetes.io/projected/8967d9da-1684-410a-a6e0-97b99e4f192c-kube-api-access-wd9gd\") pod \"perf-node-gather-daemonset-vvwzg\" (UID: \"8967d9da-1684-410a-a6e0-97b99e4f192c\") " pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.566584 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.566548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:22.706438 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:22.706284 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg"] Jun 09 11:36:22.709073 ip-10-0-130-8 kubenswrapper[2560]: W0609 11:36:22.709036 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8967d9da_1684_410a_a6e0_97b99e4f192c.slice/crio-f6cbf2c3d7b5d02205beb106c7cfd7f158127664190339fd7dcaa6b297c1dd06 WatchSource:0}: Error finding container f6cbf2c3d7b5d02205beb106c7cfd7f158127664190339fd7dcaa6b297c1dd06: Status 404 returned error can't find the container with id f6cbf2c3d7b5d02205beb106c7cfd7f158127664190339fd7dcaa6b297c1dd06 Jun 09 11:36:23.341240 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.341207 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" event={"ID":"8967d9da-1684-410a-a6e0-97b99e4f192c","Type":"ContainerStarted","Data":"bc6f56b3ea66eb6c7cff8715c6df455d6ef6f951965b6767ea142ff6392c8f13"} Jun 09 11:36:23.341670 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.341251 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" event={"ID":"8967d9da-1684-410a-a6e0-97b99e4f192c","Type":"ContainerStarted","Data":"f6cbf2c3d7b5d02205beb106c7cfd7f158127664190339fd7dcaa6b297c1dd06"} Jun 09 11:36:23.341670 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.341272 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:23.359550 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.359507 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" podStartSLOduration=1.359494062 podStartE2EDuration="1.359494062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 11:36:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 11:36:23.358603259 +0000 UTC m=+1722.175293064" watchObservedRunningTime="2026-06-09 11:36:23.359494062 +0000 UTC m=+1722.176183901" Jun 09 11:36:23.527162 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.527142 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gqdcp_1b7df2e7-4a3c-4811-9251-26743ce760ac/dns/0.log" Jun 09 11:36:23.553351 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.553330 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gqdcp_1b7df2e7-4a3c-4811-9251-26743ce760ac/kube-rbac-proxy/0.log" Jun 09 11:36:23.729445 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:23.729387 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tlcwk_d0808476-c7a5-4036-b960-6ed233a8e75a/dns-node-resolver/0.log" Jun 09 11:36:24.157280 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:24.157260 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jmkhh_437089ae-aeec-4a9f-a4af-2c1237cb1357/node-ca/0.log" Jun 09 11:36:25.269752 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:25.269727 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-d8sxj_d014bbc8-b72e-48d9-bd8b-e6d6155bdccd/serve-healthcheck-canary/0.log" Jun 09 11:36:25.715170 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:25.715145 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4slfw_25a10ba9-83f5-4263-bb59-bf7bd3e16fd4/kube-rbac-proxy/0.log" Jun 09 11:36:25.740547 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:25.740527 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4slfw_25a10ba9-83f5-4263-bb59-bf7bd3e16fd4/exporter/0.log" Jun 09 11:36:25.763619 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:25.763602 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4slfw_25a10ba9-83f5-4263-bb59-bf7bd3e16fd4/extractor/0.log" Jun 09 11:36:29.353451 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:29.353426 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vbd8c/perf-node-gather-daemonset-vvwzg" Jun 09 11:36:31.748435 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.748413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/kube-multus-additional-cni-plugins/0.log" Jun 09 11:36:31.775256 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.775236 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/egress-router-binary-copy/0.log" Jun 09 11:36:31.802798 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.802781 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/cni-plugins/0.log" Jun 09 11:36:31.826063 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.826047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/bond-cni-plugin/0.log" Jun 09 11:36:31.850378 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.850359 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/routeoverride-cni/0.log" Jun 09 11:36:31.877263 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.877241 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/whereabouts-cni-bincopy/0.log" Jun 09 11:36:31.904196 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:31.904178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-9hx4b_f2452907-e6ac-44a0-9f6b-d647c66dee6e/whereabouts-cni/0.log" Jun 09 11:36:32.340308 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:32.340283 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qcdcv_27d45ca1-bef0-4f44-985b-679862b9856d/kube-multus/0.log" Jun 09 11:36:32.456233 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:32.456206 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t6fzv_75369ceb-252a-4d5e-b532-d20e91e04137/network-metrics-daemon/0.log" Jun 09 11:36:32.478444 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:32.478422 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-t6fzv_75369ceb-252a-4d5e-b532-d20e91e04137/kube-rbac-proxy/0.log" Jun 09 11:36:33.295483 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.295457 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-controller/0.log" Jun 09 11:36:33.316933 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.316915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/1.log" Jun 09 11:36:33.317041 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.317031 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovn-acl-logging/0.log" Jun 09 11:36:33.336165 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.336145 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/kube-rbac-proxy-node/0.log" Jun 09 11:36:33.358069 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.358050 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/kube-rbac-proxy-ovn-metrics/0.log" Jun 09 11:36:33.380158 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.380140 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/northd/0.log" Jun 09 11:36:33.404496 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.404480 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/nbdb/0.log" Jun 09 11:36:33.428004 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.427990 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/sbdb/0.log" Jun 09 11:36:33.475344 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:33.475322 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78k77_f6d5f375-8e25-4cad-8288-b127c8563011/ovnkube-controller/0.log" Jun 09 11:36:34.852288 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:34.852259 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-h928v_51cdbcf1-5fae-4cb3-8c64-da0ab572a1d5/network-check-target-container/0.log" Jun 09 11:36:35.777507 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:35.777482 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-jlt76_a70f28f4-5ee0-4f15-a590-d5db13d9cec7/iptables-alerter/0.log" Jun 09 11:36:36.394708 ip-10-0-130-8 kubenswrapper[2560]: I0609 11:36:36.394685 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-gcvw4_e8f79525-22ac-4e8c-8d65-86b5d3f3fbe0/tuned/0.log"