Jun 08 12:07:11.760799 ip-10-0-128-216 systemd[1]: Starting Kubernetes Kubelet... Jun 08 12:07:12.188718 ip-10-0-128-216 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:12.188718 ip-10-0-128-216 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 12:07:12.188718 ip-10-0-128-216 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:12.188718 ip-10-0-128-216 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 12:07:12.188718 ip-10-0-128-216 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 12:07:12.189377 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.188785 2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 12:07:12.196867 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196848 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.196867 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196864 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196870 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196873 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196876 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196879 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196882 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196885 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196888 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196890 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196893 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196896 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196899 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196909 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196912 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196914 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196917 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196919 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196922 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196924 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196927 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.196935 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196930 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196933 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196935 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196938 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196941 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196943 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196946 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196949 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196952 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196955 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196957 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196966 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196969 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196972 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196974 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196977 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196980 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196991 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196994 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.197413 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196997 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.196999 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197002 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197004 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197007 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197009 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197011 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197014 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197016 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197019 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197022 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197024 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197027 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197029 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197032 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197035 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197038 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197041 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197044 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.197943 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197046 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197050 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197054 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197057 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197059 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197062 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197065 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197068 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197070 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197073 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197076 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197078 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197081 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197084 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197086 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197089 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197091 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197094 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197097 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197099 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.198403 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197102 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197104 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197106 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197109 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197111 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197115 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197118 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197120 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197123 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197126 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197128 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197131 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197134 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197136 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197139 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197142 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197144 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197147 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197149 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197151 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.198908 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197871 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197878 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197881 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197883 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197886 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197890 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197892 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197896 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197898 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197901 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197904 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197906 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197909 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197912 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197914 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197917 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197919 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197923 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197926 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197928 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.199382 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197932 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197934 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197937 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197939 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197942 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197945 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197948 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197950 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197953 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197955 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197958 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197961 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197963 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197966 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197969 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197971 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197974 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197976 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197980 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197982 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.199895 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197985 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197987 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197990 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197995 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.197999 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198002 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198006 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198009 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198012 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198015 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198017 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198020 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198023 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198026 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198029 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198033 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198035 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198038 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198041 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.200388 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198044 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198047 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198050 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198053 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198056 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198059 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198061 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198064 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198066 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198069 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198071 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198074 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198077 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198079 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198082 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198085 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198088 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198091 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198093 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198096 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.200871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198098 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198101 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198104 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198107 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198109 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198112 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198116 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198119 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198121 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198124 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198127 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198131 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198134 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198137 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198139 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198142 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198145 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198147 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198150 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198153 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.201370 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198534 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198539 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198542 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198545 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198548 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198551 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198554 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198557 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198559 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198562 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198565 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198567 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198570 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198572 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198576 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198578 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198581 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198584 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198587 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198589 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.201939 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198593 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198597 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198600 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198603 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198606 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198609 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198611 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198613 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198616 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198618 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198621 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198623 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198626 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198628 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198631 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198633 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198636 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198638 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198641 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.202422 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198644 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198646 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198648 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198651 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198654 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198656 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198659 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198665 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198668 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198671 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198673 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198676 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198679 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198682 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198685 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198688 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198690 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198695 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198697 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.202934 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198700 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198703 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198705 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198708 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198710 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198713 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198715 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198718 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198721 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198723 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198725 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198729 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198732 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198735 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198737 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198739 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198742 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198744 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198747 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.203398 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198749 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198754 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198757 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198760 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198763 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198766 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198769 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198772 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198775 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198777 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198780 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198786 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198788 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198791 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198794 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198796 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198799 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198801 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198804 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.203871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198806 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198809 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.198812 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198878 2576 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198885 2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198891 2576 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198896 2576 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198900 2576 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198903 2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198907 2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198911 2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198914 2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198918 2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198921 2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198925 2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198929 2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198932 2576 flags.go:64] FLAG: --cgroup-root="" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198934 2576 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198937 2576 flags.go:64] FLAG: --client-ca-file="" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198940 2576 flags.go:64] FLAG: --cloud-provider="external" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198943 2576 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198947 2576 flags.go:64] FLAG: --cluster-domain="" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198950 2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198953 2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 12:07:12.204330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198957 2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198961 2576 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198964 2576 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198967 2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198970 2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198974 2576 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198977 2576 flags.go:64] FLAG: --contention-profiling="false" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198980 2576 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198983 2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198986 2576 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198989 2576 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198993 2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198996 2576 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.198998 2576 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199001 2576 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199004 2576 flags.go:64] FLAG: --enable-server="true" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199007 2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199012 2576 flags.go:64] FLAG: --event-burst="100" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199015 2576 flags.go:64] FLAG: --event-qps="50" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199018 2576 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199021 2576 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199024 2576 flags.go:64] FLAG: --eviction-hard="" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199028 2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199031 2576 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199034 2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 12:07:12.204933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199037 2576 flags.go:64] FLAG: --eviction-soft="" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199040 2576 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199043 2576 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199046 2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199049 2576 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199052 2576 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199055 2576 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199059 2576 flags.go:64] FLAG: --feature-gates="" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199067 2576 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199070 2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199074 2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199077 2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199080 2576 flags.go:64] FLAG: --healthz-port="10248" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199083 2576 flags.go:64] FLAG: --help="false" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199086 2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199089 2576 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199092 2576 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199095 2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199099 2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199102 2576 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199105 2576 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199108 2576 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199111 2576 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 12:07:12.205881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199113 2576 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199117 2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199120 2576 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199123 2576 flags.go:64] FLAG: --kube-reserved="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199126 2576 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199129 2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199133 2576 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199136 2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199138 2576 flags.go:64] FLAG: --lock-file="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199141 2576 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199144 2576 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199147 2576 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199153 2576 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199155 2576 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199158 2576 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199161 2576 flags.go:64] FLAG: --logging-format="text" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199165 2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199169 2576 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199171 2576 flags.go:64] FLAG: --manifest-url="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199174 2576 flags.go:64] FLAG: --manifest-url-header="" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199179 2576 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199182 2576 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199186 2576 flags.go:64] FLAG: --max-pods="110" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199189 2576 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199191 2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 12:07:12.206919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199194 2576 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199197 2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199200 2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199203 2576 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199206 2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199213 2576 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199216 2576 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199219 2576 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199222 2576 flags.go:64] FLAG: --pod-cidr="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199225 2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199231 2576 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199234 2576 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199237 2576 flags.go:64] FLAG: --pods-per-core="0" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199240 2576 flags.go:64] FLAG: --port="10250" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199243 2576 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199246 2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a5495f6c91d69e8c" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199249 2576 flags.go:64] FLAG: --qos-reserved="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199252 2576 flags.go:64] FLAG: --read-only-port="10255" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199255 2576 flags.go:64] FLAG: --register-node="true" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199258 2576 flags.go:64] FLAG: --register-with-taints="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199261 2576 flags.go:64] FLAG: --registry-burst="10" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199264 2576 flags.go:64] FLAG: --registry-qps="5" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199267 2576 flags.go:64] FLAG: --reserved-cpus="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199271 2576 flags.go:64] FLAG: --reserved-memory="" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199274 2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 12:07:12.207846 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199278 2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199281 2576 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199283 2576 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199286 2576 flags.go:64] FLAG: --runonce="false" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199290 2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199293 2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199295 2576 flags.go:64] FLAG: --seccomp-default="false" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199298 2576 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199301 2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199304 2576 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199307 2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199310 2576 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199313 2576 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199316 2576 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199318 2576 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199321 2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199324 2576 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199327 2576 flags.go:64] FLAG: --system-cgroups="" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199331 2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199337 2576 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199339 2576 flags.go:64] FLAG: --tls-cert-file="" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199342 2576 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199347 2576 flags.go:64] FLAG: --tls-min-version="" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199350 2576 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199352 2576 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 12:07:12.208462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199355 2576 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199358 2576 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199361 2576 flags.go:64] FLAG: --v="2" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199365 2576 flags.go:64] FLAG: --version="false" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199369 2576 flags.go:64] FLAG: --vmodule="" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199374 2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199377 2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199487 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199491 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199494 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199497 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199500 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199503 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199506 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199509 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199526 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199530 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199534 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199536 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199539 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199542 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.209096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199544 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199547 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199550 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199552 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199558 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199561 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199564 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199567 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199570 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199572 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199575 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199578 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199580 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199583 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199586 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199588 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199592 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199594 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199597 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199600 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199602 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.209632 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199605 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199608 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199610 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199613 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199616 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199618 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199621 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199623 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199627 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199629 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199632 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199634 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199637 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199639 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199643 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199648 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199652 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199655 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199658 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.210138 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199661 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199664 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199667 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199670 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199672 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199675 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199677 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199682 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199686 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199689 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199692 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199695 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199698 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199701 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199703 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199706 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199709 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199712 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199715 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.210676 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199718 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199721 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199724 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199726 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199729 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199731 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199734 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199736 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199739 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199743 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199745 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199749 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199752 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199754 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199757 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199760 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199762 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199765 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199768 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199770 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.211186 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199773 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199778 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199780 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199783 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199786 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.199788 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.199794 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.208754 2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.208771 2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.208791 2576 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.208797 2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208857 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208862 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208865 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208868 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.211700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208871 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208874 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208877 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208881 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208883 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208886 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208889 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208892 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208896 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208900 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208903 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208906 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208909 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208912 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208915 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208918 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208920 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208928 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208931 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.212074 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208934 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208937 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208939 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208942 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208953 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208956 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208959 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208962 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208965 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208968 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208971 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208974 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208976 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208979 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208982 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208984 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208987 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208989 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208992 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.212549 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208994 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208997 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.208999 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209002 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209005 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209008 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209011 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209013 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209016 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209018 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209021 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209024 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209027 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209029 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209032 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209035 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209037 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209040 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209042 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209046 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209049 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.213017 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209051 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209054 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209057 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209059 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209062 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209064 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209067 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209070 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209072 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209074 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209077 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209079 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209082 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209085 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209088 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209090 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209094 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209097 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209100 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209102 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.213537 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209105 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209109 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209113 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209116 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209120 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209123 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209127 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209129 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209132 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209135 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209138 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209141 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209144 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209146 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209149 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.214030 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209152 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.209157 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209306 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209312 2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209315 2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209319 2576 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209322 2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209325 2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209328 2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209330 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209333 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209336 2576 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209339 2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209341 2576 feature_gate.go:328] unrecognized feature gate: Example Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209345 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 12:07:12.214394 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209347 2576 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209350 2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209353 2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209355 2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209358 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209360 2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209363 2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209367 2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209371 2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209374 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209376 2576 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209380 2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209384 2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209387 2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209390 2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209393 2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209396 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209399 2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209401 2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 12:07:12.214780 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209403 2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209406 2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209409 2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209411 2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209414 2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209417 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209420 2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209422 2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209425 2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209428 2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209430 2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209433 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209435 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209438 2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209442 2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209444 2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209447 2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209449 2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209452 2576 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209454 2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 12:07:12.215235 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209457 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209460 2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209462 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209464 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209467 2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209470 2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209473 2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209475 2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209478 2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209480 2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209483 2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209486 2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209488 2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209491 2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209493 2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209496 2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209499 2576 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209501 2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209504 2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209506 2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 12:07:12.215744 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209510 2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209531 2576 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209536 2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209540 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209543 2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209546 2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209550 2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209553 2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209555 2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209558 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209561 2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209564 2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209566 2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209569 2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209571 2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209574 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209577 2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209579 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209582 2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209584 2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 12:07:12.216228 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209587 2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209589 2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209592 2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209595 2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209597 2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209600 2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:12.209602 2576 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.209607 2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.209716 2576 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.212982 2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.215140 2576 server.go:1019] "Starting client certificate rotation" Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.215241 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 12:07:12.216758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.215277 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 12:07:12.217894 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.217880 2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 12:07:12.219355 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.219342 2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 12:07:12.228330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.228213 2576 log.go:25] "Validated CRI v1 runtime API" Jun 08 12:07:12.230778 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.230763 2576 log.go:25] "Validated CRI v1 image API" Jun 08 12:07:12.231641 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.231618 2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 12:07:12.236259 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.236230 2576 fs.go:135] Filesystem UUIDs: map[4032e9e8-4a5f-4393-901c-dbea5a36eaad:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 eb7485f0-7dcf-4f6b-978f-4839a4c15f9f:/dev/nvme0n1p4] Jun 08 12:07:12.236349 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.236250 2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 12:07:12.240991 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.240867 2576 manager.go:216] Machine: {Timestamp:2026-06-08 12:07:12.240342425 +0000 UTC m=+0.350598638 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100005 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2831c4543c4b09f6f1e3230e98e795 SystemUUID:ec2831c4-543c-4b09-f6f1-e3230e98e795 BootID:3d40af73-0d36-4250-ac39-6211e5ae10c7 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:b1:48:2a:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:b1:48:2a:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:a4:59:48:8b:79 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 12:07:12.240991 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.240977 2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 12:07:12.241150 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.241075 2576 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 12:07:12.244305 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.244282 2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 12:07:12.244451 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.244310 2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-216.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 12:07:12.245534 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.245509 2576 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 12:07:12.245578 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.245537 2576 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 12:07:12.245578 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.245546 2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 12:07:12.245578 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.245554 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 12:07:12.246373 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.246361 2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 12:07:12.247450 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.247438 2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 12:07:12.250382 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250370 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 08 12:07:12.250508 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250499 2576 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 12:07:12.250566 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250542 2576 kubelet.go:486] "Attempting to sync node with API server" Jun 08 12:07:12.250604 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250570 2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 12:07:12.250634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250605 2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 12:07:12.250634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250615 2576 kubelet.go:398] "Adding apiserver pod source" Jun 08 12:07:12.250634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.250628 2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 12:07:12.253149 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.253125 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 12:07:12.253225 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.253158 2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 12:07:12.258238 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.258221 2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 12:07:12.260423 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.260408 2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 12:07:12.260491 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.260437 2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 12:07:12.261469 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261458 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261473 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261479 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261484 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261493 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261498 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261506 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 12:07:12.261506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261512 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 12:07:12.261727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261531 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 12:07:12.261727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261538 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 12:07:12.261727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261546 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 12:07:12.261727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.261551 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 12:07:12.263282 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.263269 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 12:07:12.263282 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.263281 2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 12:07:12.264700 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.264668 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-216.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 12:07:12.264700 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.264665 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 12:07:12.266291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.266279 2576 server.go:1268] "Started kubelet" Jun 08 12:07:12.266497 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.266467 2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 12:07:12.266636 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.266486 2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 12:07:12.266737 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.266725 2576 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 12:07:12.267312 ip-10-0-128-216 systemd[1]: Started Kubernetes Kubelet. Jun 08 12:07:12.268241 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.268106 2576 server.go:310] "Adding debug handlers to kubelet server" Jun 08 12:07:12.268648 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.268630 2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 12:07:12.273132 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.273105 2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 12:07:12.275384 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.275366 2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-216.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 12:07:12.275506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.275493 2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 12:07:12.275506 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.275499 2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 12:07:12.275633 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.275572 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.276037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276017 2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 12:07:12.276173 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276148 2576 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 12:07:12.276287 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276216 2576 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 12:07:12.276342 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276318 2576 reconstruct.go:97] "Volume reconstruction finished" Jun 08 12:07:12.276342 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276330 2576 reconciler.go:29] "Reconciler: start to sync state" Jun 08 12:07:12.276466 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.275439 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-216.ec2.internal.18b719b28a771025 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-216.ec2.internal,UID:ip-10-0-128-216.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-216.ec2.internal,},FirstTimestamp:2026-06-08 12:07:12.266252325 +0000 UTC m=+0.376508539,LastTimestamp:2026-06-08 12:07:12.266252325 +0000 UTC m=+0.376508539,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-216.ec2.internal,}" Jun 08 12:07:12.276466 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276451 2576 factory.go:55] Registering systemd factory Jun 08 12:07:12.276595 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276479 2576 factory.go:223] Registration of the systemd container factory successfully Jun 08 12:07:12.276725 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276709 2576 factory.go:153] Registering CRI-O factory Jun 08 12:07:12.276788 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276727 2576 factory.go:223] Registration of the crio container factory successfully Jun 08 12:07:12.276788 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276785 2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 12:07:12.276887 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276809 2576 factory.go:103] Registering Raw factory Jun 08 12:07:12.276887 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.276844 2576 manager.go:1196] Started watching for new ooms in manager Jun 08 12:07:12.277820 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.277805 2576 manager.go:349] Starting recovery of all containers Jun 08 12:07:12.282944 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.282913 2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 08 12:07:12.283110 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.283080 2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-216.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 08 12:07:12.285896 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.285849 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 12:07:12.289247 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.289225 2576 manager.go:354] Recovery completed Jun 08 12:07:12.293511 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.293499 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.295215 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295193 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.295268 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295227 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.295268 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295237 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.295732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295712 2576 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 12:07:12.295732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295730 2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 12:07:12.295851 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295749 2576 state_mem.go:36] "Initialized new in-memory state store" Jun 08 12:07:12.295851 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295730 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.295954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.295860 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h7hwz" Jun 08 12:07:12.296467 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.296452 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.296591 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.296482 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.296591 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.296493 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.297264 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.297251 2576 policy_none.go:49] "None policy: Start" Jun 08 12:07:12.297264 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.297267 2576 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 12:07:12.297373 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.297277 2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 12:07:12.297501 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.297440 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-216.ec2.internal.18b719b28c30ff5d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-216.ec2.internal,UID:ip-10-0-128-216.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-216.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-216.ec2.internal,},FirstTimestamp:2026-06-08 12:07:12.295214941 +0000 UTC m=+0.405471157,LastTimestamp:2026-06-08 12:07:12.295214941 +0000 UTC m=+0.405471157,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-216.ec2.internal,}" Jun 08 12:07:12.298374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.298361 2576 policy_none.go:47] "Start" Jun 08 12:07:12.303694 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.303678 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-h7hwz" Jun 08 12:07:12.305242 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.305178 2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-216.ec2.internal.18b719b28c31425f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-216.ec2.internal,UID:ip-10-0-128-216.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-216.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-216.ec2.internal,},FirstTimestamp:2026-06-08 12:07:12.295232095 +0000 UTC m=+0.405488308,LastTimestamp:2026-06-08 12:07:12.295232095 +0000 UTC m=+0.405488308,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-216.ec2.internal,}" Jun 08 12:07:12.332130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332115 2576 manager.go:337] "Starting Device Plugin manager" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.332148 2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332161 2576 server.go:87] "Starting device plugin registration server" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332429 2576 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332442 2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332574 2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332659 2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.332668 2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.333489 2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.333572 2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.340373 2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.340409 2576 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.340429 2576 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 12:07:12.343130 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.340464 2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 12:07:12.343603 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.343296 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:12.433469 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.433442 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.435637 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.435613 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.435739 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.435649 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.435739 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.435661 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.435739 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.435684 2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.440781 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.440725 2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal"] Jun 08 12:07:12.440831 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.440796 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.442108 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.442093 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.442170 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.442125 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.442170 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.442138 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.443620 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.443608 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.443726 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.443712 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.443776 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.443745 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.444270 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444253 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.444270 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444263 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.444415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444282 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.444415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444284 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.444415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444291 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.444415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.444296 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.445543 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.445510 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.445621 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.445557 2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 12:07:12.446200 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.446165 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientMemory" Jun 08 12:07:12.446263 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.446205 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 12:07:12.446263 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.446216 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeHasSufficientPID" Jun 08 12:07:12.449751 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.449740 2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.449814 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.449760 2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-216.ec2.internal\": node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.468046 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.468022 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-216.ec2.internal\" not found" node="ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.472386 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.472372 2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-216.ec2.internal\" not found" node="ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.477557 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.477542 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.477623 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.477566 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.477623 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.477583 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7f7b943bea851ddb58741f3f1e3310a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-216.ec2.internal\" (UID: \"b7f7b943bea851ddb58741f3f1e3310a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.489985 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.489966 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.577807 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.577920 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577811 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.577920 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577841 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7f7b943bea851ddb58741f3f1e3310a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-216.ec2.internal\" (UID: \"b7f7b943bea851ddb58741f3f1e3310a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.577920 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577872 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7f7b943bea851ddb58741f3f1e3310a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-216.ec2.internal\" (UID: \"b7f7b943bea851ddb58741f3f1e3310a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.577920 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577872 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.577920 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.577887 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d90a320c156028db241c957d5edd49f7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal\" (UID: \"d90a320c156028db241c957d5edd49f7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.590895 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.590872 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.691728 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.691665 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.772148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.772114 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.777041 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:12.777023 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:12.792447 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.792426 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.892935 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.892895 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:12.993419 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:12.993343 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:13.093869 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:13.093844 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:13.194432 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:13.194405 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:13.213254 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.213231 2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:13.213386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.213374 2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 12:07:13.213539 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.213506 2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 12:07:13.290337 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:13.290312 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd90a320c156028db241c957d5edd49f7.slice/crio-da84db0c1228ca5c5247bef63d44b757c2d70f76a686d91e02fdaff9a26bb656 WatchSource:0}: Error finding container da84db0c1228ca5c5247bef63d44b757c2d70f76a686d91e02fdaff9a26bb656: Status 404 returned error can't find the container with id da84db0c1228ca5c5247bef63d44b757c2d70f76a686d91e02fdaff9a26bb656 Jun 08 12:07:13.290666 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:13.290649 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f7b943bea851ddb58741f3f1e3310a.slice/crio-2d03f6cf3cbcb6b286ef4988c0d65dd70b9873fab49ef6c2269c5b3b6cc2fc0d WatchSource:0}: Error finding container 2d03f6cf3cbcb6b286ef4988c0d65dd70b9873fab49ef6c2269c5b3b6cc2fc0d: Status 404 returned error can't find the container with id 2d03f6cf3cbcb6b286ef4988c0d65dd70b9873fab49ef6c2269c5b3b6cc2fc0d Jun 08 12:07:13.294803 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:13.294772 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:13.296176 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.296160 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:07:13.306186 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.306154 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 12:02:12 +0000 UTC" deadline="2028-01-01 06:58:24.989934889 +0000 UTC" Jun 08 12:07:13.306278 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.306204 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13722h51m11.683754524s" Jun 08 12:07:13.343816 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.343762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" event={"ID":"b7f7b943bea851ddb58741f3f1e3310a","Type":"ContainerStarted","Data":"2d03f6cf3cbcb6b286ef4988c0d65dd70b9873fab49ef6c2269c5b3b6cc2fc0d"} Jun 08 12:07:13.344685 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.344665 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" event={"ID":"d90a320c156028db241c957d5edd49f7","Type":"ContainerStarted","Data":"da84db0c1228ca5c5247bef63d44b757c2d70f76a686d91e02fdaff9a26bb656"} Jun 08 12:07:13.395161 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:13.395135 2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-216.ec2.internal\" not found" Jun 08 12:07:13.419147 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.419120 2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:13.476487 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.476463 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" Jun 08 12:07:13.484904 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.484884 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 12:07:13.486929 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.486915 2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" Jun 08 12:07:13.499886 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.499866 2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 12:07:13.744386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:13.744316 2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:14.225880 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.225686 2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 12:07:14.252066 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.252037 2576 apiserver.go:52] "Watching apiserver" Jun 08 12:07:14.260063 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.260037 2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 12:07:14.260438 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.260414 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-mg5dx","openshift-multus/multus-additional-cni-plugins-grllw","openshift-network-diagnostics/network-check-target-z2wzj","openshift-network-operator/iptables-alerter-pgtkw","openshift-ovn-kubernetes/ovnkube-node-g5z25","kube-system/global-pull-secret-syncer-wklwq","kube-system/konnectivity-agent-jgc27","kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal","openshift-cluster-node-tuning-operator/tuned-xlpvx","openshift-image-registry/node-ca-64znz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal","openshift-multus/multus-tqk8p","openshift-multus/network-metrics-daemon-ztpgp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb"] Jun 08 12:07:14.261966 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.261944 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.263245 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.263223 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.263337 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.263313 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:14.264274 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.264238 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:14.264373 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.264295 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:14.264594 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.264570 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 12:07:14.264689 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.264640 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 12:07:14.264689 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.264572 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dc5q9\"" Jun 08 12:07:14.265601 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.265585 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.267376 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.266940 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.268011 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.267947 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.268111 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.268078 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.268217 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.268180 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 12:07:14.268802 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.268435 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s9pcg\"" Jun 08 12:07:14.268802 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.268754 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.268951 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.268821 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:14.270285 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.270266 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.270383 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.270293 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.272374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.271298 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.272374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.271606 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 12:07:14.272374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.271660 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.272374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.271673 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 12:07:14.272716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.272681 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 12:07:14.272775 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.272756 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 12:07:14.272823 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.272790 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.273668 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.273648 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6s9rm\"" Jun 08 12:07:14.274593 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.274341 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.274593 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.274453 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.274767 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.274596 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9cshs\"" Jun 08 12:07:14.274819 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.274806 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jxjtx\"" Jun 08 12:07:14.275473 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.275450 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.275782 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.275757 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.275877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.275791 2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 12:07:14.276833 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.276806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278302 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278361 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278370 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-s2j2n\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278382 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278383 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278497 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.279037 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.278610 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.279490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.279196 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.279490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.279231 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bgs7g\"" Jun 08 12:07:14.279490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.279482 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 12:07:14.279766 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.279731 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.280235 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.280214 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.281160 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.281142 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6h5kb\"" Jun 08 12:07:14.281425 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.281406 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 12:07:14.281529 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.281428 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 12:07:14.281900 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.281882 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 12:07:14.282394 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.282377 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 12:07:14.282733 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.282713 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4hrgl\"" Jun 08 12:07:14.286885 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286868 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-dbus\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.286981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286897 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:14.286981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286923 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-systemd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.286981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286945 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.286981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286977 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-script-lib\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.287169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.286999 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.287169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287021 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-tuned\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.287169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287042 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7cfca38-7b21-4e52-8d90-79f59e53ba74-konnectivity-ca\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.287169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287084 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.287169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-slash\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287170 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-modprobe-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8262\" (UniqueName: \"kubernetes.io/projected/502a1100-b449-457e-9de5-7664cd9e5437-kube-api-access-l8262\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287242 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysconfig\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287290 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77147794-d0d1-404f-a3f1-ea0d2d9627c9-serviceca\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.287368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287351 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-kubernetes\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502a1100-b449-457e-9de5-7664cd9e5437-hosts-file\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7cfca38-7b21-4e52-8d90-79f59e53ba74-agent-certs\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287497 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghvg\" (UniqueName: \"kubernetes.io/projected/16af954c-1c38-473e-9697-211ec646d981-kube-api-access-9ghvg\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287551 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-system-cni-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287581 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287601 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h4tz\" (UniqueName: \"kubernetes.io/projected/b1e44096-d501-40a5-bd8a-33e19aa19e20-kube-api-access-5h4tz\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.287662 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-systemd-units\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287667 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-var-lib-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287689 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-tmp\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287712 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502a1100-b449-457e-9de5-7664cd9e5437-tmp-dir\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287756 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-netd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287781 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-os-release\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287812 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fe0067c-3d19-4070-8915-df87bb6cc277-host-slash\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287852 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-etc-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287906 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-node-log\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-config\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.287972 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/16af954c-1c38-473e-9697-211ec646d981-ovn-node-metrics-cert\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288004 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-sys\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288026 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cnibin\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288054 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-ovn\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288100 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288121 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-bin\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfwr8\" (UniqueName: \"kubernetes.io/projected/77147794-d0d1-404f-a3f1-ea0d2d9627c9-kube-api-access-wfwr8\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288176 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fe0067c-3d19-4070-8915-df87bb6cc277-iptables-alerter-script\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288211 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvgtc\" (UniqueName: \"kubernetes.io/projected/8fe0067c-3d19-4070-8915-df87bb6cc277-kube-api-access-jvgtc\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288234 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-netns\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288273 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-conf\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288294 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-run\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288299 2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288313 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-lib-modules\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288335 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-var-lib-kubelet\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288367 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-host\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288396 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xndvs\" (UniqueName: \"kubernetes.io/projected/c5c65557-a24d-4760-a660-fcc0acfb6dd7-kube-api-access-xndvs\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.288548 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288418 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-kubelet\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288438 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-log-socket\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288461 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l4qj\" (UniqueName: \"kubernetes.io/projected/f516abd9-ddbb-4687-86a2-74ffe4d295b9-kube-api-access-4l4qj\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288498 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-systemd\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288546 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77147794-d0d1-404f-a3f1-ea0d2d9627c9-host\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288580 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-env-overrides\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.289354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.288617 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-kubelet-config\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.326218 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.326196 2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-m2rg9" Jun 08 12:07:14.334751 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.334720 2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-m2rg9" Jun 08 12:07:14.377438 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.377403 2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 12:07:14.389826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389792 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-systemd-units\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.389826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389824 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-var-lib-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389849 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-device-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389904 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-var-lib-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389921 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-systemd-units\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389959 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.389986 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-bin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.390016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390010 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-daemon-config\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390031 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-tmp\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390052 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502a1100-b449-457e-9de5-7664cd9e5437-tmp-dir\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390077 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390122 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-netd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390206 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-netns\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390236 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc6th\" (UniqueName: \"kubernetes.io/projected/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-kube-api-access-tc6th\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390263 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-os-release\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.390276 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fe0067c-3d19-4070-8915-df87bb6cc277-host-slash\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.390309 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-etc-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390328 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-node-log\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390356 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-config\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.390392 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:14.890368581 +0000 UTC m=+3.000624782 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390423 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/502a1100-b449-457e-9de5-7664cd9e5437-tmp-dir\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390429 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-etc-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390439 2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390449 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-os-release\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390447 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8fe0067c-3d19-4070-8915-df87bb6cc277-host-slash\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390480 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/16af954c-1c38-473e-9697-211ec646d981-ovn-node-metrics-cert\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390482 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-netd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390447 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-node-log\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390506 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-etc-kubernetes\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390560 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-sys\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390582 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cnibin\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-ovn\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.390801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9fs\" (UniqueName: \"kubernetes.io/projected/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kube-api-access-9b9fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390657 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-system-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390679 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-kubelet\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390626 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-sys\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390702 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-hostroot\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390742 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390766 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390767 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-bin\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390805 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-cni-bin\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390759 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-ovn\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390807 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-socket-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390853 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-conf-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390884 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cnibin\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.390937 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfwr8\" (UniqueName: \"kubernetes.io/projected/77147794-d0d1-404f-a3f1-ea0d2d9627c9-kube-api-access-wfwr8\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391008 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-config\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391082 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fe0067c-3d19-4070-8915-df87bb6cc277-iptables-alerter-script\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391108 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvgtc\" (UniqueName: \"kubernetes.io/projected/8fe0067c-3d19-4070-8915-df87bb6cc277-kube-api-access-jvgtc\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.391619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391176 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-netns\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391218 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-multus\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391246 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-conf\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391286 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-run\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391285 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391311 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-lib-modules\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391330 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-netns\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391378 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-run\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391417 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-conf\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391436 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-var-lib-kubelet\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391486 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-lib-modules\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391550 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-var-lib-kubelet\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391565 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-host\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391619 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8fe0067c-3d19-4070-8915-df87bb6cc277-iptables-alerter-script\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391488 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-host\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391849 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xndvs\" (UniqueName: \"kubernetes.io/projected/c5c65557-a24d-4760-a660-fcc0acfb6dd7-kube-api-access-xndvs\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391877 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-kubelet\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-log-socket\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.391997 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l4qj\" (UniqueName: \"kubernetes.io/projected/f516abd9-ddbb-4687-86a2-74ffe4d295b9-kube-api-access-4l4qj\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392020 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-systemd\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392040 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77147794-d0d1-404f-a3f1-ea0d2d9627c9-host\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392065 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-env-overrides\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392085 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-kubelet-config\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392112 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-kubelet\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392124 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/77147794-d0d1-404f-a3f1-ea0d2d9627c9-host\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392141 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-log-socket\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392158 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-systemd\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392143 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-dbus\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392200 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392205 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-kubelet-config\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392226 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-multus-certs\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392234 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-dbus\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392255 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392283 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-systemd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.393918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392307 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392333 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-script-lib\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392352 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-systemd\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392356 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-os-release\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392379 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cni-binary-copy\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392402 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392403 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-run-openvswitch\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392446 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-tuned\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392473 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7cfca38-7b21-4e52-8d90-79f59e53ba74-konnectivity-ca\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392500 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392531 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysctl-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392540 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-slash\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392564 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-env-overrides\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392566 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-registration-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392607 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-sys-fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392635 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-socket-dir-parent\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392658 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.394570 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392686 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cnibin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392713 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-modprobe-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392736 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8262\" (UniqueName: \"kubernetes.io/projected/502a1100-b449-457e-9de5-7664cd9e5437-kube-api-access-l8262\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392758 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392780 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysconfig\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392808 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77147794-d0d1-404f-a3f1-ea0d2d9627c9-serviceca\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392833 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392857 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392881 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-kubernetes\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392899 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/16af954c-1c38-473e-9697-211ec646d981-ovnkube-script-lib\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392924 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502a1100-b449-457e-9de5-7664cd9e5437-hosts-file\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392958 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7cfca38-7b21-4e52-8d90-79f59e53ba74-agent-certs\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.392989 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-slash\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393005 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghvg\" (UniqueName: \"kubernetes.io/projected/16af954c-1c38-473e-9697-211ec646d981-kube-api-access-9ghvg\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-k8s-cni-cncf-io\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393066 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-system-cni-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393095 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7cfca38-7b21-4e52-8d90-79f59e53ba74-konnectivity-ca\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.395291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393100 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393104 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393037 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-run-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h4tz\" (UniqueName: \"kubernetes.io/projected/b1e44096-d501-40a5-bd8a-33e19aa19e20-kube-api-access-5h4tz\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393175 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f516abd9-ddbb-4687-86a2-74ffe4d295b9-system-cni-dir\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393379 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-sysconfig\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.393436 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393549 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-kubernetes\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393583 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/502a1100-b449-457e-9de5-7664cd9e5437-hosts-file\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.393641 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:14.893623945 +0000 UTC m=+3.003880159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393784 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/16af954c-1c38-473e-9697-211ec646d981-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393823 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/77147794-d0d1-404f-a3f1-ea0d2d9627c9-serviceca\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393874 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-modprobe-d\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.393969 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-tmp\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.394296 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f516abd9-ddbb-4687-86a2-74ffe4d295b9-cni-binary-copy\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.394680 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/16af954c-1c38-473e-9697-211ec646d981-ovn-node-metrics-cert\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.395402 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c5c65557-a24d-4760-a660-fcc0acfb6dd7-etc-tuned\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.395934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.395608 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7cfca38-7b21-4e52-8d90-79f59e53ba74-agent-certs\") pod \"konnectivity-agent-jgc27\" (UID: \"a7cfca38-7b21-4e52-8d90-79f59e53ba74\") " pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.403245 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.403227 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:14.403372 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.403249 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:14.403372 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.403261 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:14.403372 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.403326 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:14.903310629 +0000 UTC m=+3.013566829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:14.405317 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.405289 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvgtc\" (UniqueName: \"kubernetes.io/projected/8fe0067c-3d19-4070-8915-df87bb6cc277-kube-api-access-jvgtc\") pod \"iptables-alerter-pgtkw\" (UID: \"8fe0067c-3d19-4070-8915-df87bb6cc277\") " pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.406265 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.406223 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h4tz\" (UniqueName: \"kubernetes.io/projected/b1e44096-d501-40a5-bd8a-33e19aa19e20-kube-api-access-5h4tz\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.406743 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.406720 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l4qj\" (UniqueName: \"kubernetes.io/projected/f516abd9-ddbb-4687-86a2-74ffe4d295b9-kube-api-access-4l4qj\") pod \"multus-additional-cni-plugins-grllw\" (UID: \"f516abd9-ddbb-4687-86a2-74ffe4d295b9\") " pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.406835 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.406741 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghvg\" (UniqueName: \"kubernetes.io/projected/16af954c-1c38-473e-9697-211ec646d981-kube-api-access-9ghvg\") pod \"ovnkube-node-g5z25\" (UID: \"16af954c-1c38-473e-9697-211ec646d981\") " pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.407007 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.406971 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xndvs\" (UniqueName: \"kubernetes.io/projected/c5c65557-a24d-4760-a660-fcc0acfb6dd7-kube-api-access-xndvs\") pod \"tuned-xlpvx\" (UID: \"c5c65557-a24d-4760-a660-fcc0acfb6dd7\") " pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.407096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.407051 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfwr8\" (UniqueName: \"kubernetes.io/projected/77147794-d0d1-404f-a3f1-ea0d2d9627c9-kube-api-access-wfwr8\") pod \"node-ca-64znz\" (UID: \"77147794-d0d1-404f-a3f1-ea0d2d9627c9\") " pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.407468 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.407446 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8262\" (UniqueName: \"kubernetes.io/projected/502a1100-b449-457e-9de5-7664cd9e5437-kube-api-access-l8262\") pod \"node-resolver-mg5dx\" (UID: \"502a1100-b449-457e-9de5-7664cd9e5437\") " pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.494372 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494340 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-registration-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494372 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494377 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-sys-fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494397 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-socket-dir-parent\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494431 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494452 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cnibin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494482 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-k8s-cni-cncf-io\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494488 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-registration-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494508 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-device-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494565 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-bin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494580 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-sys-fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.494609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494607 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-k8s-cni-cncf-io\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cnibin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494637 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-socket-dir-parent\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494638 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-bin\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494657 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-daemon-config\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494663 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-etc-selinux\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494683 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494684 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-device-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494690 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-netns\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494722 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-netns\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494727 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc6th\" (UniqueName: \"kubernetes.io/projected/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-kube-api-access-tc6th\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494766 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-etc-kubernetes\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494794 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9fs\" (UniqueName: \"kubernetes.io/projected/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kube-api-access-9b9fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494812 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-etc-kubernetes\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494816 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-system-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494848 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-kubelet\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-system-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494871 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-hostroot\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495051 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494900 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-socket-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494922 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-hostroot\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494903 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-kubelet\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494941 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-conf-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494972 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-multus\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.494977 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-conf-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495007 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495015 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-var-lib-cni-multus\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495028 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-socket-dir\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-multus-certs\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495066 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-host-run-multus-certs\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495071 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-cni-dir\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495090 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-os-release\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495115 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cni-binary-copy\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495169 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-os-release\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495229 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-multus-daemon-config\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.495755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.495562 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-cni-binary-copy\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.502958 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.502929 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9fs\" (UniqueName: \"kubernetes.io/projected/b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8-kube-api-access-9b9fs\") pod \"aws-ebs-csi-driver-node-jm2qb\" (UID: \"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.503074 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.502932 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc6th\" (UniqueName: \"kubernetes.io/projected/6d0ab198-453d-4179-b1c1-78eca2c0fa8d-kube-api-access-tc6th\") pod \"multus-tqk8p\" (UID: \"6d0ab198-453d-4179-b1c1-78eca2c0fa8d\") " pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.574559 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.574443 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:14.584617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.584599 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pgtkw" Jun 08 12:07:14.594832 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.594809 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:14.600171 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.600150 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" Jun 08 12:07:14.607987 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.607970 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-mg5dx" Jun 08 12:07:14.616436 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.616416 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-grllw" Jun 08 12:07:14.624039 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.624021 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-64znz" Jun 08 12:07:14.630545 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.630529 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" Jun 08 12:07:14.636175 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.636158 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tqk8p" Jun 08 12:07:14.898084 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.898002 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:14.898084 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.898053 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:14.898297 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.898185 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:14.898297 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.898188 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:14.898297 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.898237 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:15.898223636 +0000 UTC m=+4.008479835 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:14.898297 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.898250 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:15.898244317 +0000 UTC m=+4.008500517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:14.998965 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:14.998922 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:14.999151 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.999105 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:14.999151 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.999131 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:14.999151 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.999143 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:14.999312 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:14.999202 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:15.999188785 +0000 UTC m=+4.109444988 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:15.148689 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.148624 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7cfca38_7b21_4e52_8d90_79f59e53ba74.slice/crio-000fff214c9ca9b3f0ac620e86eb791f162d815918b75b672dc3dbb5462442cb WatchSource:0}: Error finding container 000fff214c9ca9b3f0ac620e86eb791f162d815918b75b672dc3dbb5462442cb: Status 404 returned error can't find the container with id 000fff214c9ca9b3f0ac620e86eb791f162d815918b75b672dc3dbb5462442cb Jun 08 12:07:15.149788 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.149764 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod502a1100_b449_457e_9de5_7664cd9e5437.slice/crio-c7dda90f0c6323b636abc09e9b17bb9154f43e37bc10829d24ac89a826146639 WatchSource:0}: Error finding container c7dda90f0c6323b636abc09e9b17bb9154f43e37bc10829d24ac89a826146639: Status 404 returned error can't find the container with id c7dda90f0c6323b636abc09e9b17bb9154f43e37bc10829d24ac89a826146639 Jun 08 12:07:15.150858 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.150835 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c65557_a24d_4760_a660_fcc0acfb6dd7.slice/crio-fe61f4c97d17f9b232265f97fc603dcd0dc79b8506458b7117cb2b7ae212e66f WatchSource:0}: Error finding container fe61f4c97d17f9b232265f97fc603dcd0dc79b8506458b7117cb2b7ae212e66f: Status 404 returned error can't find the container with id fe61f4c97d17f9b232265f97fc603dcd0dc79b8506458b7117cb2b7ae212e66f Jun 08 12:07:15.152436 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.152408 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb30cfc5a_8ab9_4fe4_bc5b_55a3af0d5dd8.slice/crio-c3c2a1547c345100f883631316a7e8ad533d95e36f48332b44713fef379b5349 WatchSource:0}: Error finding container c3c2a1547c345100f883631316a7e8ad533d95e36f48332b44713fef379b5349: Status 404 returned error can't find the container with id c3c2a1547c345100f883631316a7e8ad533d95e36f48332b44713fef379b5349 Jun 08 12:07:15.154248 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.153954 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77147794_d0d1_404f_a3f1_ea0d2d9627c9.slice/crio-521f4bebd8d92babe6372d2cb55e2b72c5d6b77d0e5168d435e1ed0b72e87377 WatchSource:0}: Error finding container 521f4bebd8d92babe6372d2cb55e2b72c5d6b77d0e5168d435e1ed0b72e87377: Status 404 returned error can't find the container with id 521f4bebd8d92babe6372d2cb55e2b72c5d6b77d0e5168d435e1ed0b72e87377 Jun 08 12:07:15.156180 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.156160 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf516abd9_ddbb_4687_86a2_74ffe4d295b9.slice/crio-ad892dd40290b850c1cf887115562eaf59d4aa6def3b3166618edd3653e9bb94 WatchSource:0}: Error finding container ad892dd40290b850c1cf887115562eaf59d4aa6def3b3166618edd3653e9bb94: Status 404 returned error can't find the container with id ad892dd40290b850c1cf887115562eaf59d4aa6def3b3166618edd3653e9bb94 Jun 08 12:07:15.157602 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.157581 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fe0067c_3d19_4070_8915_df87bb6cc277.slice/crio-0ed9d638c9ff7c5dab4098eed4640ce2fa5de3accbe79b2f774866bd29501428 WatchSource:0}: Error finding container 0ed9d638c9ff7c5dab4098eed4640ce2fa5de3accbe79b2f774866bd29501428: Status 404 returned error can't find the container with id 0ed9d638c9ff7c5dab4098eed4640ce2fa5de3accbe79b2f774866bd29501428 Jun 08 12:07:15.158659 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.158609 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16af954c_1c38_473e_9697_211ec646d981.slice/crio-8ffbfd8e832bedda4ecff4dc48ce8a86c06af1ae85cce17d577e4747cfe35544 WatchSource:0}: Error finding container 8ffbfd8e832bedda4ecff4dc48ce8a86c06af1ae85cce17d577e4747cfe35544: Status 404 returned error can't find the container with id 8ffbfd8e832bedda4ecff4dc48ce8a86c06af1ae85cce17d577e4747cfe35544 Jun 08 12:07:15.159475 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:15.159421 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0ab198_453d_4179_b1c1_78eca2c0fa8d.slice/crio-f6b22dff73fbc2200aa97562f3936977e0563938311a23bbee93f840b18abb3d WatchSource:0}: Error finding container f6b22dff73fbc2200aa97562f3936977e0563938311a23bbee93f840b18abb3d: Status 404 returned error can't find the container with id f6b22dff73fbc2200aa97562f3936977e0563938311a23bbee93f840b18abb3d Jun 08 12:07:15.336945 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.336888 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 12:02:14 +0000 UTC" deadline="2027-12-29 17:13:28.644241198 +0000 UTC" Jun 08 12:07:15.336945 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.336941 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13661h6m13.307304481s" Jun 08 12:07:15.341173 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.341148 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:15.341322 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:15.341276 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:15.348819 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.348791 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" event={"ID":"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8","Type":"ContainerStarted","Data":"c3c2a1547c345100f883631316a7e8ad533d95e36f48332b44713fef379b5349"} Jun 08 12:07:15.350061 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.350034 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" event={"ID":"c5c65557-a24d-4760-a660-fcc0acfb6dd7","Type":"ContainerStarted","Data":"fe61f4c97d17f9b232265f97fc603dcd0dc79b8506458b7117cb2b7ae212e66f"} Jun 08 12:07:15.351141 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.351120 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tqk8p" event={"ID":"6d0ab198-453d-4179-b1c1-78eca2c0fa8d","Type":"ContainerStarted","Data":"f6b22dff73fbc2200aa97562f3936977e0563938311a23bbee93f840b18abb3d"} Jun 08 12:07:15.352325 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.352306 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pgtkw" event={"ID":"8fe0067c-3d19-4070-8915-df87bb6cc277","Type":"ContainerStarted","Data":"0ed9d638c9ff7c5dab4098eed4640ce2fa5de3accbe79b2f774866bd29501428"} Jun 08 12:07:15.353587 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.353561 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerStarted","Data":"ad892dd40290b850c1cf887115562eaf59d4aa6def3b3166618edd3653e9bb94"} Jun 08 12:07:15.354885 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.354851 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mg5dx" event={"ID":"502a1100-b449-457e-9de5-7664cd9e5437","Type":"ContainerStarted","Data":"c7dda90f0c6323b636abc09e9b17bb9154f43e37bc10829d24ac89a826146639"} Jun 08 12:07:15.356080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.356034 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgc27" event={"ID":"a7cfca38-7b21-4e52-8d90-79f59e53ba74","Type":"ContainerStarted","Data":"000fff214c9ca9b3f0ac620e86eb791f162d815918b75b672dc3dbb5462442cb"} Jun 08 12:07:15.360358 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.360297 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" event={"ID":"b7f7b943bea851ddb58741f3f1e3310a","Type":"ContainerStarted","Data":"38e90aa6bd1d105f301793e8f606adf8afebd2aeea843f69f9c0849138cbcb6e"} Jun 08 12:07:15.361544 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.361501 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"8ffbfd8e832bedda4ecff4dc48ce8a86c06af1ae85cce17d577e4747cfe35544"} Jun 08 12:07:15.362617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.362596 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-64znz" event={"ID":"77147794-d0d1-404f-a3f1-ea0d2d9627c9","Type":"ContainerStarted","Data":"521f4bebd8d92babe6372d2cb55e2b72c5d6b77d0e5168d435e1ed0b72e87377"} Jun 08 12:07:15.374818 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.374767 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-216.ec2.internal" podStartSLOduration=2.374755654 podStartE2EDuration="2.374755654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:15.374287807 +0000 UTC m=+3.484544028" watchObservedRunningTime="2026-06-08 12:07:15.374755654 +0000 UTC m=+3.485011876" Jun 08 12:07:15.906119 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.906037 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:15.906290 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:15.906132 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:15.906290 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:15.906273 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:15.906391 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:15.906335 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:17.906318169 +0000 UTC m=+6.016574372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:15.906768 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:15.906745 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:15.906848 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:15.906807 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:17.906792756 +0000 UTC m=+6.017048959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:16.007330 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.007291 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:16.007545 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.007477 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:16.007545 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.007498 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:16.007545 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.007511 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:16.007719 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.007581 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:18.007563695 +0000 UTC m=+6.117819898 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:16.338120 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.337988 2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 12:02:14 +0000 UTC" deadline="2027-11-29 15:45:12.692035817 +0000 UTC" Jun 08 12:07:16.338120 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.338047 2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12939h37m56.353992532s" Jun 08 12:07:16.343151 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.342633 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:16.343151 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.342767 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:16.343566 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.343420 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:16.343566 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:16.343531 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:16.381608 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.380328 2576 generic.go:356] "Generic (PLEG): container finished" podID="d90a320c156028db241c957d5edd49f7" containerID="fed9563afb6a984c30f1a82b64150861a169db8f9bd7d93d8a32505d3639c7f9" exitCode=0 Jun 08 12:07:16.381608 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:16.381319 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" event={"ID":"d90a320c156028db241c957d5edd49f7","Type":"ContainerDied","Data":"fed9563afb6a984c30f1a82b64150861a169db8f9bd7d93d8a32505d3639c7f9"} Jun 08 12:07:17.341294 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:17.340918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:17.341294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:17.341047 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:17.410537 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:17.410487 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" event={"ID":"d90a320c156028db241c957d5edd49f7","Type":"ContainerStarted","Data":"37aed9899be89ff98473eb0b2ab0937d4cf186e3df906066f156d381a5d26c56"} Jun 08 12:07:17.426350 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:17.426295 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-216.ec2.internal" podStartSLOduration=4.426274503 podStartE2EDuration="4.426274503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:17.425960777 +0000 UTC m=+5.536217001" watchObservedRunningTime="2026-06-08 12:07:17.426274503 +0000 UTC m=+5.536530726" Jun 08 12:07:17.924875 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:17.924834 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:17.925059 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:17.924901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:17.925059 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:17.925050 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:17.925161 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:17.925111 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:21.925091646 +0000 UTC m=+10.035347865 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:17.925559 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:17.925538 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:17.925707 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:17.925601 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:21.925584373 +0000 UTC m=+10.035840575 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:18.025810 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:18.025729 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:18.025989 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.025951 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:18.025989 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.025975 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:18.025989 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.025989 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:18.026154 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.026052 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:22.026032649 +0000 UTC m=+10.136288856 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:18.341361 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:18.341284 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:18.341816 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.341424 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:18.342171 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:18.342128 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:18.342273 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:18.342248 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:19.341340 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:19.340864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:19.341340 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:19.341007 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:20.341010 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:20.340960 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:20.341180 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:20.341029 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:20.341180 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:20.341148 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:20.341311 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:20.341283 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:21.341634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:21.341598 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:21.342092 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:21.341745 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:21.963345 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:21.963434 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:21.963568 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:21.963626 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:29.96360832 +0000 UTC m=+18.073864536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:21.963692 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:21.964092 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:21.963723 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:29.963711854 +0000 UTC m=+18.073968068 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:22.064241 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:22.064205 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:22.064428 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.064381 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:22.064428 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.064407 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:22.064428 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.064419 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:22.064653 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.064485 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:30.064465559 +0000 UTC m=+18.174721760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:22.343115 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:22.342867 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:22.343115 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:22.342973 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:22.343115 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.342993 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:22.343115 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:22.343031 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:23.341459 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:23.341070 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:23.341459 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:23.341208 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:24.341258 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:24.341224 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:24.341733 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:24.341224 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:24.341733 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:24.341363 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:24.341733 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:24.341441 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:25.341363 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:25.341330 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:25.341831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:25.341461 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:26.340879 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:26.340806 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:26.341056 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:26.340959 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:26.341056 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:26.341017 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:26.341162 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:26.341131 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:27.341355 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:27.341320 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:27.341782 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:27.341429 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:28.341824 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:28.341787 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:28.342249 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:28.341836 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:28.342249 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:28.341908 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:28.342249 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:28.341991 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:29.341199 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:29.341162 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:29.341476 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:29.341277 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:30.023676 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:30.023632 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:30.024147 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:30.023689 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:30.024147 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.023795 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:30.024147 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.023802 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:30.024147 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.023862 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:46.023848177 +0000 UTC m=+34.134104377 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:30.024147 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.023878 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:07:46.023870714 +0000 UTC m=+34.134126914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:30.124849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:30.124815 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:30.125052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.125005 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:30.125052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.125025 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:30.125052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.125035 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:30.125183 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.125091 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:07:46.125072063 +0000 UTC m=+34.235328262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:30.341101 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:30.341019 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:30.341276 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:30.341019 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:30.341276 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.341157 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:30.341276 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:30.341242 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:31.341754 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:31.341704 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:31.342234 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:31.341867 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:32.341996 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:32.341949 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:32.342453 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:32.342014 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:32.342453 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:32.342090 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:32.342453 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:32.342197 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:33.340819 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:33.340782 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:33.341019 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:33.340901 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:34.341102 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.340931 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:34.341607 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.340973 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:34.341607 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:34.341244 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:34.341607 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:34.341339 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:34.448199 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.448170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"e1f97f6fbdca69181f2f2952b95e7f3d21cad6372d244d327845ee87e26853cd"} Jun 08 12:07:34.448290 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.448241 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"9b05950baa04ab05b61d6a2559ea3436a9eb965303aea3a6df0a78705063f7b6"} Jun 08 12:07:34.449463 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.449443 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-64znz" event={"ID":"77147794-d0d1-404f-a3f1-ea0d2d9627c9","Type":"ContainerStarted","Data":"e8fb5f624ba9afaadb89854ac9fcf42cf938afa2034d08f94486639c62d31d72"} Jun 08 12:07:34.450804 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.450786 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" event={"ID":"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8","Type":"ContainerStarted","Data":"52adc51ebafd5ef769ee453049534b6bda0edae2e296795d8be02fbd34a3dc23"} Jun 08 12:07:34.452049 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.452029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" event={"ID":"c5c65557-a24d-4760-a660-fcc0acfb6dd7","Type":"ContainerStarted","Data":"f2d3007dacbacc134af0d0fd8faa090c1e6f4fff71abe7bd453aa5df0432c731"} Jun 08 12:07:34.453395 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.453377 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tqk8p" event={"ID":"6d0ab198-453d-4179-b1c1-78eca2c0fa8d","Type":"ContainerStarted","Data":"ed289696d5b4cc84e5f7ab189b666507cccb153cdbe82aeb834ebd108ca1eea7"} Jun 08 12:07:34.454729 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.454707 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="8d5f66eabc52ee7d535e3988ea61def4d0be00058a4795da20b309195ba1bcde" exitCode=0 Jun 08 12:07:34.454820 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.454795 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"8d5f66eabc52ee7d535e3988ea61def4d0be00058a4795da20b309195ba1bcde"} Jun 08 12:07:34.456219 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.456195 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-mg5dx" event={"ID":"502a1100-b449-457e-9de5-7664cd9e5437","Type":"ContainerStarted","Data":"81b4e2b3d94fe14ad7871e61163d2e3d551d9230b649ad2fb6618d0368c5485e"} Jun 08 12:07:34.457575 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.457555 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgc27" event={"ID":"a7cfca38-7b21-4e52-8d90-79f59e53ba74","Type":"ContainerStarted","Data":"8979adf8221087a4aaa788c695cd350ade01d4aa3d6b63ed06d06bcaec352768"} Jun 08 12:07:34.466114 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.466066 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-64znz" podStartSLOduration=12.028945245 podStartE2EDuration="22.466047712s" totalImagesPullingTime="10.437102467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:34.465619711 +0000 UTC m=+22.575875932" watchObservedRunningTime="2026-06-08 12:07:34.466047712 +0000 UTC m=+22.576303934" Jun 08 12:07:34.484030 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.483986 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xlpvx" podStartSLOduration=3.603185742 podStartE2EDuration="22.483971642s" totalImagesPullingTime="18.8807859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:34.483349917 +0000 UTC m=+22.593606150" watchObservedRunningTime="2026-06-08 12:07:34.483971642 +0000 UTC m=+22.594227863" Jun 08 12:07:34.526260 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.526204 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-mg5dx" podStartSLOduration=3.646107134 podStartE2EDuration="22.526187236s" totalImagesPullingTime="18.880080102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:34.50132098 +0000 UTC m=+22.611577202" watchObservedRunningTime="2026-06-08 12:07:34.526187236 +0000 UTC m=+22.636443457" Jun 08 12:07:34.554650 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.554603 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tqk8p" podStartSLOduration=3.665920358 podStartE2EDuration="22.554586441s" totalImagesPullingTime="18.888666083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:34.55458205 +0000 UTC m=+22.664838272" watchObservedRunningTime="2026-06-08 12:07:34.554586441 +0000 UTC m=+22.664842662" Jun 08 12:07:34.571812 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:34.571764 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jgc27" podStartSLOduration=3.6894969939999998 podStartE2EDuration="22.571750558s" totalImagesPullingTime="18.882253564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:34.571216705 +0000 UTC m=+22.681472924" watchObservedRunningTime="2026-06-08 12:07:34.571750558 +0000 UTC m=+22.682006779" Jun 08 12:07:35.341440 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.341363 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:35.341877 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:35.341500 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:35.460821 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.460792 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pgtkw" event={"ID":"8fe0067c-3d19-4070-8915-df87bb6cc277","Type":"ContainerStarted","Data":"cbc31a7936c3e48b337d91c7ad9f6aafac3f9dd2124663ff092dd66760ca2714"} Jun 08 12:07:35.463952 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.463923 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"276ef9b2288bdf043bd652c7d0fa1020952581a3109332cda8a128566eb92544"} Jun 08 12:07:35.464050 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.463991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"76816bc7b6ae2f73c65f22cec5c86ef7ebcda101f75f13ee27293d7e888af159"} Jun 08 12:07:35.464050 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.464032 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"d699883026d43e8ec8194e7641a3ae558a8173dcd2a3c01ab2324fbf2eea7b32"} Jun 08 12:07:35.464125 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.464061 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"26a06ff47313ab154aa9383f351067e103151d2fa9a634767800fc6fbd2f638c"} Jun 08 12:07:35.482816 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.482769 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pgtkw" podStartSLOduration=4.609541654 podStartE2EDuration="23.482748198s" totalImagesPullingTime="18.873206544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:35.482549035 +0000 UTC m=+23.592805257" watchObservedRunningTime="2026-06-08 12:07:35.482748198 +0000 UTC m=+23.593004434" Jun 08 12:07:35.596669 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:35.596597 2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 12:07:36.341418 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.341361 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:36.341418 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.341396 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:36.342164 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:36.341508 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:36.342164 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:36.341624 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:36.343439 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.343350 2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T12:07:35.596616952Z","UUID":"c4b9768e-434c-45e9-a883-01fc310a02c7","Handler":null,"Name":"","Endpoint":""} Jun 08 12:07:36.345427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.345405 2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 12:07:36.345557 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.345447 2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 12:07:36.470144 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:36.469895 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" event={"ID":"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8","Type":"ContainerStarted","Data":"b1fa0c9fcf52a65c2b7abe700519c6657048fdc34b5a016a3df8139c7a7d2786"} Jun 08 12:07:37.340736 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.340644 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:37.340946 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:37.340788 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:37.475749 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.475711 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"ef0dbc9004c24365a917e98760aafed3d65f658d895c5416d5fad710846298da"} Jun 08 12:07:37.477745 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.477721 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" event={"ID":"b30cfc5a-8ab9-4fe4-bc5b-55a3af0d5dd8","Type":"ContainerStarted","Data":"7815220453e87462b57b04842becc148ffb09254fb2c4b8029bc610d662fc602"} Jun 08 12:07:37.496817 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.496776 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jm2qb" podStartSLOduration=4.056992318 podStartE2EDuration="25.496760838s" totalImagesPullingTime="21.43976852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:37.496444695 +0000 UTC m=+25.606700928" watchObservedRunningTime="2026-06-08 12:07:37.496760838 +0000 UTC m=+25.607017060" Jun 08 12:07:37.668139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.668046 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:37.668997 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:37.668934 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:38.340747 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:38.340717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:38.340943 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:38.340717 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:38.340943 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:38.340852 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:38.340943 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:38.340927 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:39.341032 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.340854 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:39.341702 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:39.341120 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:39.486461 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.486424 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="e744063d3632d61e3e1ac728f1d845daa1b852c1042dc8c59991bdf71036b4cc" exitCode=0 Jun 08 12:07:39.486655 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.486476 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"e744063d3632d61e3e1ac728f1d845daa1b852c1042dc8c59991bdf71036b4cc"} Jun 08 12:07:39.490410 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.490385 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" event={"ID":"16af954c-1c38-473e-9697-211ec646d981","Type":"ContainerStarted","Data":"e3540176818acbc3ec5857d06d1a4e23c02a93992605dfb018f53b3d24065ae6"} Jun 08 12:07:39.490703 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.490686 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:39.505460 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:39.505437 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:40.340902 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.340864 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:40.341162 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:40.341033 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:40.341162 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.341079 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:40.341590 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:40.341172 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:40.493080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.493050 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:40.493230 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.493133 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:40.508124 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.508098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:07:40.541945 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:40.541894 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" podStartSLOduration=9.578939072 podStartE2EDuration="28.541879097s" totalImagesPullingTime="18.962940025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:39.554405071 +0000 UTC m=+27.664661307" watchObservedRunningTime="2026-06-08 12:07:40.541879097 +0000 UTC m=+28.652135319" Jun 08 12:07:41.137569 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.137532 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2wzj"] Jun 08 12:07:41.137813 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.137662 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:41.137813 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:41.137776 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:41.141191 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.141163 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wklwq"] Jun 08 12:07:41.141316 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.141283 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:41.141396 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:41.141379 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:41.141837 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.141816 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ztpgp"] Jun 08 12:07:41.141921 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.141909 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:41.142026 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:41.142010 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:41.496549 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.496452 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="786e1a00cc8024f63187606eb9754b0c1102c70836a2e71c191193c33cbe4754" exitCode=0 Jun 08 12:07:41.496883 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.496554 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"786e1a00cc8024f63187606eb9754b0c1102c70836a2e71c191193c33cbe4754"} Jun 08 12:07:41.900619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.900534 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:41.900785 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.900715 2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 08 12:07:41.901175 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:41.901158 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jgc27" Jun 08 12:07:42.341764 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:42.341674 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:42.341912 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:42.341764 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:42.341912 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:42.341769 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:42.341912 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:42.341863 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:43.341339 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:43.341167 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:43.341741 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:43.341419 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:43.504024 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:43.503950 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="fb6a8d8b0efc9c8a39e3fc5d62381510b7941d0d7c598266cd1e0e18d1221b45" exitCode=0 Jun 08 12:07:43.504024 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:43.504007 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"fb6a8d8b0efc9c8a39e3fc5d62381510b7941d0d7c598266cd1e0e18d1221b45"} Jun 08 12:07:44.340743 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:44.340703 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:44.340949 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:44.340807 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:44.340949 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:44.340863 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:44.341062 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:44.340964 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:45.340604 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:45.340575 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:45.341259 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:45.340696 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:46.042732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:46.042692 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:46.042928 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:46.042745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:46.042928 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.042871 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:46.042928 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.042874 2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:46.043090 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.042934 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:18.042915035 +0000 UTC m=+66.153171249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 12:07:46.043090 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.042952 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret podName:03c699ff-b0c6-44af-bd0a-8b8679dbf4c9 nodeName:}" failed. No retries permitted until 2026-06-08 12:08:18.042943196 +0000 UTC m=+66.153199398 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret") pod "global-pull-secret-syncer-wklwq" (UID: "03c699ff-b0c6-44af-bd0a-8b8679dbf4c9") : object "kube-system"/"original-pull-secret" not registered Jun 08 12:07:46.143876 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:46.143838 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:46.144071 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.144011 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 12:07:46.144071 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.144032 2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 12:07:46.144071 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.144046 2576 projected.go:196] Error preparing data for projected volume kube-api-access-2tmw4 for pod openshift-network-diagnostics/network-check-target-z2wzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:46.144216 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.144113 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4 podName:b7f15cc3-634e-4fe3-bf78-d5edc451a6aa nodeName:}" failed. No retries permitted until 2026-06-08 12:08:18.144093746 +0000 UTC m=+66.254349962 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2tmw4" (UniqueName: "kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4") pod "network-check-target-z2wzj" (UID: "b7f15cc3-634e-4fe3-bf78-d5edc451a6aa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 12:07:46.340866 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:46.340777 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:46.340866 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:46.340822 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:46.341385 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.340926 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z2wzj" podUID="b7f15cc3-634e-4fe3-bf78-d5edc451a6aa" Jun 08 12:07:46.341385 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:46.341016 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wklwq" podUID="03c699ff-b0c6-44af-bd0a-8b8679dbf4c9" Jun 08 12:07:47.341401 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.341368 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:47.341900 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.341567 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:07:47.679645 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.679558 2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-216.ec2.internal" event="NodeReady" Jun 08 12:07:47.679812 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.679728 2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 12:07:47.713977 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.713945 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:07:47.716358 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.716331 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.719072 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.719046 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 12:07:47.719072 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.719059 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jwnkf\"" Jun 08 12:07:47.719263 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.719074 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 12:07:47.719263 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.719136 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 12:07:47.725388 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.725367 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 12:07:47.730081 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.730057 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:07:47.732128 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.732107 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cmk7n"] Jun 08 12:07:47.734018 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.733995 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.735646 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.735627 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bv5jn"] Jun 08 12:07:47.736923 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.736886 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2hkrn\"" Jun 08 12:07:47.737109 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.737090 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 12:07:47.737212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.737094 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 12:07:47.737360 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.737343 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 12:07:47.737448 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.737408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.739700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.739678 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6km9q\"" Jun 08 12:07:47.740165 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.739898 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 12:07:47.740165 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.740021 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 12:07:47.744009 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.743972 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cmk7n"] Jun 08 12:07:47.748295 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.748269 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bv5jn"] Jun 08 12:07:47.856798 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856759 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.856798 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856803 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-config-volume\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856885 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9lq4\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856933 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856955 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-tmp-dir\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.856979 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857035 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857091 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvvs4\" (UniqueName: \"kubernetes.io/projected/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-kube-api-access-hvvs4\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857139 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857163 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-cert\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857204 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857244 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqz7\" (UniqueName: \"kubernetes.io/projected/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-kube-api-access-9hqz7\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.857327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.857268 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958122 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958039 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvvs4\" (UniqueName: \"kubernetes.io/projected/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-kube-api-access-hvvs4\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.958122 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958107 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958133 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-cert\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958163 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958182 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqz7\" (UniqueName: \"kubernetes.io/projected/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-kube-api-access-9hqz7\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958202 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958242 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958269 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-config-volume\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.958307 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958318 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9lq4\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958347 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958368 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-tmp-dir\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.958388 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:07:48.458367217 +0000 UTC m=+36.568623434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:07:47.958432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958431 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958481 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958700 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.958708 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-tmp-dir\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.958798 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.958813 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:47.958860 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:07:48.458843176 +0000 UTC m=+36.569099397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:07:47.959052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.959050 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.959395 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.959278 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-config-volume\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.959395 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.959322 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.962932 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.962902 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.962932 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.962920 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-cert\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.963121 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.962939 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.969504 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.969478 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvvs4\" (UniqueName: \"kubernetes.io/projected/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-kube-api-access-hvvs4\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:47.970170 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.970124 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqz7\" (UniqueName: \"kubernetes.io/projected/22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2-kube-api-access-9hqz7\") pod \"ingress-canary-cmk7n\" (UID: \"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2\") " pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:47.970754 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.970724 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9lq4\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:47.974509 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:47.974486 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:48.044885 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.044840 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cmk7n" Jun 08 12:07:48.341025 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.340985 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:07:48.341195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.340984 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:07:48.344048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.344020 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 12:07:48.344748 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.344488 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 12:07:48.345104 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.345084 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qfftc\"" Jun 08 12:07:48.345212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.345106 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 12:07:48.463486 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.463439 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:48.463675 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.463534 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:48.463675 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:48.463611 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:48.463675 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:48.463654 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:07:48.463675 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:48.463669 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:07:48.463856 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:48.463679 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:07:49.463661683 +0000 UTC m=+37.573917886 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:07:48.463856 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:48.463733 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:07:49.463716353 +0000 UTC m=+37.573972568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:07:48.751429 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:48.751247 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cmk7n"] Jun 08 12:07:48.755386 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:48.755358 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a1a1d0_cbd6_4d4b_8ef3_8b6f16791af2.slice/crio-fa9444d32459535507b35bfd85f26029f94acbbba830fea1feedd8bcfda7b5b5 WatchSource:0}: Error finding container fa9444d32459535507b35bfd85f26029f94acbbba830fea1feedd8bcfda7b5b5: Status 404 returned error can't find the container with id fa9444d32459535507b35bfd85f26029f94acbbba830fea1feedd8bcfda7b5b5 Jun 08 12:07:49.340926 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.340888 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:07:49.343781 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.343753 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 12:07:49.343925 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.343794 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gr2v8\"" Jun 08 12:07:49.471824 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.471786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:49.471938 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:49.472003 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:07:51.471984836 +0000 UTC m=+39.582241055 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.472024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:49.472176 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:49.472189 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:07:49.472257 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:49.472223 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:07:51.472214174 +0000 UTC m=+39.582470398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:07:49.520034 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.519999 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cmk7n" event={"ID":"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2","Type":"ContainerStarted","Data":"fa9444d32459535507b35bfd85f26029f94acbbba830fea1feedd8bcfda7b5b5"} Jun 08 12:07:49.523405 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.523378 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="68ff0c7c0d81a411025235d5eb42173542c42f7c42690221271d5949d66819a2" exitCode=0 Jun 08 12:07:49.523553 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:49.523417 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"68ff0c7c0d81a411025235d5eb42173542c42f7c42690221271d5949d66819a2"} Jun 08 12:07:50.528661 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:50.528628 2576 generic.go:356] "Generic (PLEG): container finished" podID="f516abd9-ddbb-4687-86a2-74ffe4d295b9" containerID="277679d139f5f8a9f55282f72a1ce5de0ca3f6ab18c2acee538cde8852bccb2b" exitCode=0 Jun 08 12:07:50.529013 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:50.528670 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerDied","Data":"277679d139f5f8a9f55282f72a1ce5de0ca3f6ab18c2acee538cde8852bccb2b"} Jun 08 12:07:51.486627 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.486360 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:51.486627 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.486601 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:51.486831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:51.486538 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:51.486831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:51.486692 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:07:51.486831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:51.486700 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:07:55.486684134 +0000 UTC m=+43.596940334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:07:51.486831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:51.486703 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:07:51.486831 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:51.486743 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:07:55.486732435 +0000 UTC m=+43.596988634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:07:51.534069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.534033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grllw" event={"ID":"f516abd9-ddbb-4687-86a2-74ffe4d295b9","Type":"ContainerStarted","Data":"87c78be94e22874326fce9ec41e9c378e43c301bc723d9803427a0161d6666da"} Jun 08 12:07:51.535511 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.535489 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cmk7n" event={"ID":"22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2","Type":"ContainerStarted","Data":"eef91b4e84a85b82e3f24bd4a491dade8518a2e95a7d9f17dc2d66482ec129cd"} Jun 08 12:07:51.562314 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.562264 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-grllw" podStartSLOduration=8.594081902 podStartE2EDuration="39.562247383s" totalImagesPullingTime="30.968165481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:51.560640195 +0000 UTC m=+39.670896416" watchObservedRunningTime="2026-06-08 12:07:51.562247383 +0000 UTC m=+39.672503605" Jun 08 12:07:51.578089 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:51.578043 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cmk7n" podStartSLOduration=2.134991686 podStartE2EDuration="4.578026995s" totalImagesPullingTime="2.443035309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:07:51.577366535 +0000 UTC m=+39.687622758" watchObservedRunningTime="2026-06-08 12:07:51.578026995 +0000 UTC m=+39.688283216" Jun 08 12:07:55.517005 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:55.516968 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:55.517033 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:55.517125 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:55.517189 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:08:03.517172659 +0000 UTC m=+51.627429253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:55.517189 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:55.517208 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:07:55.517360 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:07:55.517253 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:08:03.517238705 +0000 UTC m=+51.627494905 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:07:57.328792 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.328758 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl"] Jun 08 12:07:57.347864 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.347835 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl"] Jun 08 12:07:57.348014 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.347951 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.350729 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.350707 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 08 12:07:57.351612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351592 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 12:07:57.351738 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351616 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 12:07:57.351738 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351621 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 08 12:07:57.351738 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351635 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 08 12:07:57.351738 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351694 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 12:07:57.351738 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.351720 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 08 12:07:57.531918 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.531878 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f7fab3-e91b-4775-95cb-52967acbd1b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.532091 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.531937 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.532091 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.531987 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.532091 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.532022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.532091 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.532067 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t77tj\" (UniqueName: \"kubernetes.io/projected/a8f7fab3-e91b-4775-95cb-52967acbd1b2-kube-api-access-t77tj\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.532224 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.532118 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.632985 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.632893 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t77tj\" (UniqueName: \"kubernetes.io/projected/a8f7fab3-e91b-4775-95cb-52967acbd1b2-kube-api-access-t77tj\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.632985 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.632955 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.633164 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.633024 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f7fab3-e91b-4775-95cb-52967acbd1b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.633164 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.633110 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.633164 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.633136 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.633164 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.633153 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.633681 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.633661 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/a8f7fab3-e91b-4775-95cb-52967acbd1b2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.637322 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.637290 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-ca\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.637439 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.637343 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.637439 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.637403 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.637580 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.637447 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/a8f7fab3-e91b-4775-95cb-52967acbd1b2-hub\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.641013 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.640992 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t77tj\" (UniqueName: \"kubernetes.io/projected/a8f7fab3-e91b-4775-95cb-52967acbd1b2-kube-api-access-t77tj\") pod \"cluster-proxy-proxy-agent-b66cb8d-9fbjl\" (UID: \"a8f7fab3-e91b-4775-95cb-52967acbd1b2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.672348 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.672311 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:07:57.823464 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:57.823431 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl"] Jun 08 12:07:57.826737 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:07:57.826712 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8f7fab3_e91b_4775_95cb_52967acbd1b2.slice/crio-0cc4f399e6240ec87da59ac4fbe8c7c678c76ecf7334205705e60a2963aa60aa WatchSource:0}: Error finding container 0cc4f399e6240ec87da59ac4fbe8c7c678c76ecf7334205705e60a2963aa60aa: Status 404 returned error can't find the container with id 0cc4f399e6240ec87da59ac4fbe8c7c678c76ecf7334205705e60a2963aa60aa Jun 08 12:07:58.552995 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:07:58.552957 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerStarted","Data":"0cc4f399e6240ec87da59ac4fbe8c7c678c76ecf7334205705e60a2963aa60aa"} Jun 08 12:08:02.564588 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:02.564554 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerStarted","Data":"4bae82d51b549b653999198075b4b3ac613aa2daf8ac3d9e660a23522f25181d"} Jun 08 12:08:03.569414 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:03.569372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerStarted","Data":"c916dfc45ca57cc7b47e7d96e02a3df3fa2793ac47d847a2cbc1a2beae3bc220"} Jun 08 12:08:03.569805 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:03.569426 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerStarted","Data":"a43c964e3cdae78877f9d454cfc8fc0dc26f60e334751748e576fc9d35186b6c"} Jun 08 12:08:03.576456 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:03.576423 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:08:03.576643 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:03.576499 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:08:03.576643 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:03.576588 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:03.576643 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:03.576606 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:08:03.576643 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:03.576618 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:08:03.576863 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:03.576669 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:08:19.576647101 +0000 UTC m=+67.686903305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:08:03.576863 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:03.576691 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:08:19.576681726 +0000 UTC m=+67.686937931 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:08:03.588573 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:03.588487 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" podStartSLOduration=2.050599411 podStartE2EDuration="6.588469294s" totalImagesPullingTime="4.537869883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:03.587356623 +0000 UTC m=+51.697612846" watchObservedRunningTime="2026-06-08 12:08:03.588469294 +0000 UTC m=+51.698725515" Jun 08 12:08:12.510126 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:12.510097 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g5z25" Jun 08 12:08:18.081158 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.081116 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:08:18.081630 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.081172 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:08:18.083721 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.083701 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 12:08:18.083771 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.083706 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 12:08:18.091812 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:18.091786 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 12:08:18.091890 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:18.091877 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:09:22.091848287 +0000 UTC m=+130.202104501 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : secret "metrics-daemon-secret" not found Jun 08 12:08:18.094011 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.093990 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03c699ff-b0c6-44af-bd0a-8b8679dbf4c9-original-pull-secret\") pod \"global-pull-secret-syncer-wklwq\" (UID: \"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9\") " pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:08:18.181821 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.181784 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:08:18.184724 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.184704 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 12:08:18.195140 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.195108 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 12:08:18.205609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.205586 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tmw4\" (UniqueName: \"kubernetes.io/projected/b7f15cc3-634e-4fe3-bf78-d5edc451a6aa-kube-api-access-2tmw4\") pod \"network-check-target-z2wzj\" (UID: \"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa\") " pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:08:18.355897 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.355817 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qfftc\"" Jun 08 12:08:18.359057 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.359036 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wklwq" Jun 08 12:08:18.364187 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.364166 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:08:18.494400 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.494366 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wklwq"] Jun 08 12:08:18.497844 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:08:18.497807 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c699ff_b0c6_44af_bd0a_8b8679dbf4c9.slice/crio-c005d4a711a159b59a8768d79ec78e28efefa5de3fc3b4d25bcfe8e6d3fbce60 WatchSource:0}: Error finding container c005d4a711a159b59a8768d79ec78e28efefa5de3fc3b4d25bcfe8e6d3fbce60: Status 404 returned error can't find the container with id c005d4a711a159b59a8768d79ec78e28efefa5de3fc3b4d25bcfe8e6d3fbce60 Jun 08 12:08:18.508428 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.508398 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z2wzj"] Jun 08 12:08:18.511700 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:08:18.511664 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f15cc3_634e_4fe3_bf78_d5edc451a6aa.slice/crio-8b4325f186470364377a1277f90d934e74002b4e2190f29e698a2a8754621fc5 WatchSource:0}: Error finding container 8b4325f186470364377a1277f90d934e74002b4e2190f29e698a2a8754621fc5: Status 404 returned error can't find the container with id 8b4325f186470364377a1277f90d934e74002b4e2190f29e698a2a8754621fc5 Jun 08 12:08:18.606730 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.606637 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wklwq" event={"ID":"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9","Type":"ContainerStarted","Data":"c005d4a711a159b59a8768d79ec78e28efefa5de3fc3b4d25bcfe8e6d3fbce60"} Jun 08 12:08:18.607713 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:18.607694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2wzj" event={"ID":"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa","Type":"ContainerStarted","Data":"8b4325f186470364377a1277f90d934e74002b4e2190f29e698a2a8754621fc5"} Jun 08 12:08:19.594085 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:19.594049 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:19.594102 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:19.594220 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:19.594239 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:19.594217 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:19.594301 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:08:51.59428589 +0000 UTC m=+99.704542093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:08:19.594554 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:19.594316 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:08:51.594311171 +0000 UTC m=+99.704567370 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:08:23.624923 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:23.624875 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wklwq" event={"ID":"03c699ff-b0c6-44af-bd0a-8b8679dbf4c9","Type":"ContainerStarted","Data":"cf0ed6329c20cfb69c02f0b5029886e48c1afbfe8ac575b32f1223d9db8f4385"} Jun 08 12:08:23.626470 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:23.626442 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z2wzj" event={"ID":"b7f15cc3-634e-4fe3-bf78-d5edc451a6aa","Type":"ContainerStarted","Data":"618731a5cbfc2cc5522800ac9280fdd003e7db45c207e74ab9d9c7a22acd369a"} Jun 08 12:08:23.626609 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:23.626562 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:08:23.640222 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:23.640173 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wklwq" podStartSLOduration=66.739564647 podStartE2EDuration="1m11.640160209s" totalImagesPullingTime="4.900595562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:23.639858794 +0000 UTC m=+71.750115028" watchObservedRunningTime="2026-06-08 12:08:23.640160209 +0000 UTC m=+71.750416430" Jun 08 12:08:23.653836 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:23.653733 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z2wzj" podStartSLOduration=66.770049751 podStartE2EDuration="1m11.653715731s" totalImagesPullingTime="4.88366598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:08:23.653574434 +0000 UTC m=+71.763830656" watchObservedRunningTime="2026-06-08 12:08:23.653715731 +0000 UTC m=+71.763971953" Jun 08 12:08:51.645787 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:51.645747 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:51.645854 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:51.645923 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:51.645948 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:51.645980 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:51.646032 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:09:55.646009276 +0000 UTC m=+163.756265480 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:08:51.646294 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:08:51.646053 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:09:55.646043071 +0000 UTC m=+163.756299274 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:08:54.631691 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:08:54.631660 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z2wzj" Jun 08 12:09:22.159639 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:09:22.159588 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:09:22.160126 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:22.159745 2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 12:09:22.160126 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:22.159825 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs podName:b1e44096-d501-40a5-bd8a-33e19aa19e20 nodeName:}" failed. No retries permitted until 2026-06-08 12:11:24.159807652 +0000 UTC m=+252.270063856 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs") pod "network-metrics-daemon-ztpgp" (UID: "b1e44096-d501-40a5-bd8a-33e19aa19e20") : secret "metrics-daemon-secret" not found Jun 08 12:09:50.728274 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:50.728236 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" Jun 08 12:09:50.753091 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:50.753055 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bv5jn" podUID="a9ef63cf-e8c1-427a-8376-18b4dc9d6cce" Jun 08 12:09:50.864467 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:09:50.864435 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:09:50.864660 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:09:50.864435 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:09:52.350144 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:52.350109 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-ztpgp" podUID="b1e44096-d501-40a5-bd8a-33e19aa19e20" Jun 08 12:09:55.691251 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:09:55.691212 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:09:55.691259 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:55.691357 2576 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:55.691369 2576 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-667ff6457b-c86qv: secret "image-registry-tls" not found Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:55.691433 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls podName:a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb nodeName:}" failed. No retries permitted until 2026-06-08 12:11:57.691414632 +0000 UTC m=+285.801670833 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls") pod "image-registry-667ff6457b-c86qv" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb") : secret "image-registry-tls" not found Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:55.691463 2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 12:09:55.691652 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:09:55.691511 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls podName:a9ef63cf-e8c1-427a-8376-18b4dc9d6cce nodeName:}" failed. No retries permitted until 2026-06-08 12:11:57.691498383 +0000 UTC m=+285.801754587 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls") pod "dns-default-bv5jn" (UID: "a9ef63cf-e8c1-427a-8376-18b4dc9d6cce") : secret "dns-default-metrics-tls" not found Jun 08 12:10:03.341129 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:10:03.341030 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:10:47.203045 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:10:47.203017 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mg5dx_502a1100-b449-457e-9de5-7664cd9e5437/dns-node-resolver/0.log" Jun 08 12:10:48.005834 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:10:48.005807 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-64znz_77147794-d0d1-404f-a3f1-ea0d2d9627c9/node-ca/0.log" Jun 08 12:10:49.002921 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:10:49.002897 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cmk7n_22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2/serve-healthcheck-canary/0.log" Jun 08 12:11:09.610047 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.610013 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nclf6"] Jun 08 12:11:09.612153 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.612132 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.615838 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.615817 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 12:11:09.616958 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.616943 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 12:11:09.616958 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.616951 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5b5xn\"" Jun 08 12:11:09.617165 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.616975 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 12:11:09.619374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.619356 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 12:11:09.629508 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.629483 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nclf6"] Jun 08 12:11:09.712982 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.712950 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/272c8051-3716-48df-919b-d975fb099487-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.713139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.712997 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/272c8051-3716-48df-919b-d975fb099487-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.713139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.713053 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/272c8051-3716-48df-919b-d975fb099487-crio-socket\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.713139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.713076 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/272c8051-3716-48df-919b-d975fb099487-data-volume\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.713139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.713136 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t4mw\" (UniqueName: \"kubernetes.io/projected/272c8051-3716-48df-919b-d975fb099487-kube-api-access-5t4mw\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.813826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813786 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/272c8051-3716-48df-919b-d975fb099487-crio-socket\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.813826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813828 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/272c8051-3716-48df-919b-d975fb099487-data-volume\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813869 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t4mw\" (UniqueName: \"kubernetes.io/projected/272c8051-3716-48df-919b-d975fb099487-kube-api-access-5t4mw\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813901 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/272c8051-3716-48df-919b-d975fb099487-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813928 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/272c8051-3716-48df-919b-d975fb099487-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.813928 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/272c8051-3716-48df-919b-d975fb099487-crio-socket\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814312 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.814288 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/272c8051-3716-48df-919b-d975fb099487-data-volume\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.814602 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.814583 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/272c8051-3716-48df-919b-d975fb099487-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.816244 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.816224 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/272c8051-3716-48df-919b-d975fb099487-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.822242 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.822216 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t4mw\" (UniqueName: \"kubernetes.io/projected/272c8051-3716-48df-919b-d975fb099487-kube-api-access-5t4mw\") pod \"insights-runtime-extractor-nclf6\" (UID: \"272c8051-3716-48df-919b-d975fb099487\") " pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:09.921040 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:09.920953 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nclf6" Jun 08 12:11:10.041060 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:10.041028 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nclf6"] Jun 08 12:11:10.045980 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:10.045954 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272c8051_3716_48df_919b_d975fb099487.slice/crio-affbbaac16a99934153a97c33b1235ad5f94475331b018c9ac1a0b5395bdc186 WatchSource:0}: Error finding container affbbaac16a99934153a97c33b1235ad5f94475331b018c9ac1a0b5395bdc186: Status 404 returned error can't find the container with id affbbaac16a99934153a97c33b1235ad5f94475331b018c9ac1a0b5395bdc186 Jun 08 12:11:10.083508 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:10.083482 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nclf6" event={"ID":"272c8051-3716-48df-919b-d975fb099487","Type":"ContainerStarted","Data":"affbbaac16a99934153a97c33b1235ad5f94475331b018c9ac1a0b5395bdc186"} Jun 08 12:11:11.088017 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:11.087936 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nclf6" event={"ID":"272c8051-3716-48df-919b-d975fb099487","Type":"ContainerStarted","Data":"21ec37ef8f5ed46c82678d03e8b5b7d27ddd9c6b306d74a5093174fd8198ac08"} Jun 08 12:11:11.088017 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:11.087991 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nclf6" event={"ID":"272c8051-3716-48df-919b-d975fb099487","Type":"ContainerStarted","Data":"367c2dad15d401f84f0fd923bb6fb61d333ab20febabd15c0b8eba6b86b576b0"} Jun 08 12:11:13.096437 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:13.096400 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nclf6" event={"ID":"272c8051-3716-48df-919b-d975fb099487","Type":"ContainerStarted","Data":"b90eaeae34f4dc56473abbebf53a9775da2cd4602d2615d572afe12a2c68d456"} Jun 08 12:11:13.115006 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:13.114941 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nclf6" podStartSLOduration=1.572238074 podStartE2EDuration="4.114925857s" totalImagesPullingTime="2.542687783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:11:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:13.113971532 +0000 UTC m=+241.224227755" watchObservedRunningTime="2026-06-08 12:11:13.114925857 +0000 UTC m=+241.225182112" Jun 08 12:11:14.335116 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.335075 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz"] Jun 08 12:11:14.337386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.337363 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:14.339961 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.339933 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 08 12:11:14.340062 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.340000 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dgcv5\"" Jun 08 12:11:14.346347 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.346325 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz"] Jun 08 12:11:14.450736 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.450696 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22faebf6-91aa-4907-8dbd-4512eb96777e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-vw9fz\" (UID: \"22faebf6-91aa-4907-8dbd-4512eb96777e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:14.551203 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.551167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22faebf6-91aa-4907-8dbd-4512eb96777e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-vw9fz\" (UID: \"22faebf6-91aa-4907-8dbd-4512eb96777e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:14.553703 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.553680 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/22faebf6-91aa-4907-8dbd-4512eb96777e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5b6b8f594-vw9fz\" (UID: \"22faebf6-91aa-4907-8dbd-4512eb96777e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:14.647634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.647541 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:14.772285 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:14.772251 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz"] Jun 08 12:11:14.775259 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:14.775232 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22faebf6_91aa_4907_8dbd_4512eb96777e.slice/crio-3619f2c64b2554f932779e89a499ed91e9c51d3acf62fc8336de19abdf52fe89 WatchSource:0}: Error finding container 3619f2c64b2554f932779e89a499ed91e9c51d3acf62fc8336de19abdf52fe89: Status 404 returned error can't find the container with id 3619f2c64b2554f932779e89a499ed91e9c51d3acf62fc8336de19abdf52fe89 Jun 08 12:11:15.103015 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:15.102983 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" event={"ID":"22faebf6-91aa-4907-8dbd-4512eb96777e","Type":"ContainerStarted","Data":"3619f2c64b2554f932779e89a499ed91e9c51d3acf62fc8336de19abdf52fe89"} Jun 08 12:11:16.107402 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:16.107314 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" event={"ID":"22faebf6-91aa-4907-8dbd-4512eb96777e","Type":"ContainerStarted","Data":"dc639d9f67c238b66779588a7741cabcf474c44bb2faad172c84499284bc06a1"} Jun 08 12:11:16.107810 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:16.107548 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:16.112196 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:16.112175 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" Jun 08 12:11:16.122970 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:16.122930 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5b6b8f594-vw9fz" podStartSLOduration=1.078019714 podStartE2EDuration="2.122917745s" totalImagesPullingTime="1.044898031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:16.121921082 +0000 UTC m=+244.232177329" watchObservedRunningTime="2026-06-08 12:11:16.122917745 +0000 UTC m=+244.233173967" Jun 08 12:11:20.756409 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.756375 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wl44h"] Jun 08 12:11:20.758382 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.758366 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.761121 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.761094 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz"] Jun 08 12:11:20.761397 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.761347 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 12:11:20.761533 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.761494 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 12:11:20.762071 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.762049 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 12:11:20.762667 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.762653 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p49nf\"" Jun 08 12:11:20.763076 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.763056 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.764556 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.764538 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 12:11:20.764709 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.764694 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 12:11:20.765231 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.765217 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 12:11:20.766709 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.766681 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 08 12:11:20.766709 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.766691 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 08 12:11:20.766842 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.766733 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 08 12:11:20.766842 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.766776 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sgxc4\"" Jun 08 12:11:20.780784 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.780762 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz"] Jun 08 12:11:20.790413 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790384 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.790567 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790427 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.790567 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790445 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-tls\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790567 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790462 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-wtmp\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790567 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790505 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-metrics-client-ca\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790563 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5296e584-d40e-4cbb-aa50-c581fceed528-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790595 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-textfile\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrgbk\" (UniqueName: \"kubernetes.io/projected/5296e584-d40e-4cbb-aa50-c581fceed528-kube-api-access-xrgbk\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790678 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-sys\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790700 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-root\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790718 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frkzr\" (UniqueName: \"kubernetes.io/projected/94bd54c1-8bae-4f80-ac32-72ce49b79474-kube-api-access-frkzr\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790736 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790771 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.790934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.790934 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.790823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891337 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891301 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891337 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891336 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-tls\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891359 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-wtmp\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891380 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-metrics-client-ca\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891443 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5296e584-d40e-4cbb-aa50-c581fceed528-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-textfile\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891558 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrgbk\" (UniqueName: \"kubernetes.io/projected/5296e584-d40e-4cbb-aa50-c581fceed528-kube-api-access-xrgbk\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-sys\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891625 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-wtmp\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891628 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-root\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891675 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-sys\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891684 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frkzr\" (UniqueName: \"kubernetes.io/projected/94bd54c1-8bae-4f80-ac32-72ce49b79474-kube-api-access-frkzr\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891714 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891745 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891774 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891823 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891850 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.891888 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/94bd54c1-8bae-4f80-ac32-72ce49b79474-root\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.892366 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.891905 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/5296e584-d40e-4cbb-aa50-c581fceed528-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.892366 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:11:20.891972 2576 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 08 12:11:20.892366 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.892049 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-metrics-client-ca\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.892366 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:11:20.892137 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls podName:5296e584-d40e-4cbb-aa50-c581fceed528 nodeName:}" failed. No retries permitted until 2026-06-08 12:11:21.392119062 +0000 UTC m=+249.502375268 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-wk4hz" (UID: "5296e584-d40e-4cbb-aa50-c581fceed528") : secret "kube-state-metrics-tls" not found Jun 08 12:11:20.892366 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.892134 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.892624 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.892430 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-textfile\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.892624 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.892459 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.892692 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.892661 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5296e584-d40e-4cbb-aa50-c581fceed528-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.894178 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.894159 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-tls\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.894473 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.894454 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/94bd54c1-8bae-4f80-ac32-72ce49b79474-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:20.894599 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.894582 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.900380 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.900354 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrgbk\" (UniqueName: \"kubernetes.io/projected/5296e584-d40e-4cbb-aa50-c581fceed528-kube-api-access-xrgbk\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:20.900495 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:20.900423 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frkzr\" (UniqueName: \"kubernetes.io/projected/94bd54c1-8bae-4f80-ac32-72ce49b79474-kube-api-access-frkzr\") pod \"node-exporter-wl44h\" (UID: \"94bd54c1-8bae-4f80-ac32-72ce49b79474\") " pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:21.069608 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.069576 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wl44h" Jun 08 12:11:21.079447 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:21.079414 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94bd54c1_8bae_4f80_ac32_72ce49b79474.slice/crio-f4d208dcd4e77dccddeafecfc21d9c9128b3f77c2085a596bf379029aebe9fc5 WatchSource:0}: Error finding container f4d208dcd4e77dccddeafecfc21d9c9128b3f77c2085a596bf379029aebe9fc5: Status 404 returned error can't find the container with id f4d208dcd4e77dccddeafecfc21d9c9128b3f77c2085a596bf379029aebe9fc5 Jun 08 12:11:21.125200 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.125165 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl44h" event={"ID":"94bd54c1-8bae-4f80-ac32-72ce49b79474","Type":"ContainerStarted","Data":"f4d208dcd4e77dccddeafecfc21d9c9128b3f77c2085a596bf379029aebe9fc5"} Jun 08 12:11:21.395540 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.395432 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:21.397805 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.397776 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5296e584-d40e-4cbb-aa50-c581fceed528-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk4hz\" (UID: \"5296e584-d40e-4cbb-aa50-c581fceed528\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:21.674325 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.674220 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" Jun 08 12:11:21.826935 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:21.826901 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz"] Jun 08 12:11:21.831468 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:21.831436 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5296e584_d40e_4cbb_aa50_c581fceed528.slice/crio-b0a1a53d79760422cbe3c383f01e5043dba00bb3fdecf126a5a449042dd7446c WatchSource:0}: Error finding container b0a1a53d79760422cbe3c383f01e5043dba00bb3fdecf126a5a449042dd7446c: Status 404 returned error can't find the container with id b0a1a53d79760422cbe3c383f01e5043dba00bb3fdecf126a5a449042dd7446c Jun 08 12:11:22.129076 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:22.129038 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" event={"ID":"5296e584-d40e-4cbb-aa50-c581fceed528","Type":"ContainerStarted","Data":"b0a1a53d79760422cbe3c383f01e5043dba00bb3fdecf126a5a449042dd7446c"} Jun 08 12:11:23.133468 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:23.133436 2576 generic.go:356] "Generic (PLEG): container finished" podID="94bd54c1-8bae-4f80-ac32-72ce49b79474" containerID="306b0926266a129fef5313ae664caa77c98c040941bbd69655a5f3eacddcdc7a" exitCode=0 Jun 08 12:11:23.133865 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:23.133533 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl44h" event={"ID":"94bd54c1-8bae-4f80-ac32-72ce49b79474","Type":"ContainerDied","Data":"306b0926266a129fef5313ae664caa77c98c040941bbd69655a5f3eacddcdc7a"} Jun 08 12:11:23.135140 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:23.135112 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" event={"ID":"5296e584-d40e-4cbb-aa50-c581fceed528","Type":"ContainerStarted","Data":"b046ff28a4b0a9fecd6fe9289fb5c50b5263e69870aeaa76cd151f53baf1f112"} Jun 08 12:11:24.140183 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.140153 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl44h" event={"ID":"94bd54c1-8bae-4f80-ac32-72ce49b79474","Type":"ContainerStarted","Data":"beef3c2c1612e1d67a3dc9993134ec26fa6121da41cbd535462be85161dbdd63"} Jun 08 12:11:24.140636 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.140207 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl44h" event={"ID":"94bd54c1-8bae-4f80-ac32-72ce49b79474","Type":"ContainerStarted","Data":"bc467ca4c1978a7c9d4bdf4baa73a8d68e46282af8dca80ffd2a43ec5abc5753"} Jun 08 12:11:24.142143 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.142120 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" event={"ID":"5296e584-d40e-4cbb-aa50-c581fceed528","Type":"ContainerStarted","Data":"044fc1d82a6059e6e66277a2d4958b237bb5ca13298e3cf177a6fc149bedb069"} Jun 08 12:11:24.142273 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.142166 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" event={"ID":"5296e584-d40e-4cbb-aa50-c581fceed528","Type":"ContainerStarted","Data":"077d2855a790d1000015a0d358e06bcbaeee0d21478dff7e500e53c502258da5"} Jun 08 12:11:24.163854 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.163805 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wl44h" podStartSLOduration=3.050432103 podStartE2EDuration="4.163791185s" totalImagesPullingTime="1.049919683s" totalInitContainerRuntime="63.439399ms" isStatefulPod=true podCreationTimestamp="2026-06-08 12:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:24.162557282 +0000 UTC m=+252.272813506" watchObservedRunningTime="2026-06-08 12:11:24.163791185 +0000 UTC m=+252.274047406" Jun 08 12:11:24.196281 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.196234 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk4hz" podStartSLOduration=2.952262537 podStartE2EDuration="4.196218416s" totalImagesPullingTime="1.243955879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:24.196140901 +0000 UTC m=+252.306397122" watchObservedRunningTime="2026-06-08 12:11:24.196218416 +0000 UTC m=+252.306474616" Jun 08 12:11:24.214523 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.214492 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:11:24.216898 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.216860 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1e44096-d501-40a5-bd8a-33e19aa19e20-metrics-certs\") pod \"network-metrics-daemon-ztpgp\" (UID: \"b1e44096-d501-40a5-bd8a-33e19aa19e20\") " pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:11:24.344697 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.344666 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gr2v8\"" Jun 08 12:11:24.352929 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.352908 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ztpgp" Jun 08 12:11:24.476713 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:24.476683 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ztpgp"] Jun 08 12:11:24.479479 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:24.479451 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e44096_d501_40a5_bd8a_33e19aa19e20.slice/crio-1eabd6d7d3597ac0669f3b2dd220fec80cb2363dd707123d941c9cf6b3814d4a WatchSource:0}: Error finding container 1eabd6d7d3597ac0669f3b2dd220fec80cb2363dd707123d941c9cf6b3814d4a: Status 404 returned error can't find the container with id 1eabd6d7d3597ac0669f3b2dd220fec80cb2363dd707123d941c9cf6b3814d4a Jun 08 12:11:25.147876 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.147835 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztpgp" event={"ID":"b1e44096-d501-40a5-bd8a-33e19aa19e20","Type":"ContainerStarted","Data":"1eabd6d7d3597ac0669f3b2dd220fec80cb2363dd707123d941c9cf6b3814d4a"} Jun 08 12:11:25.960751 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.960716 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9f8649bc-jprlp"] Jun 08 12:11:25.963148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.963122 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:25.965731 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.965708 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-pzmvt\"" Jun 08 12:11:25.965879 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.965770 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 08 12:11:25.965879 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.965769 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 08 12:11:25.965988 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.965946 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 08 12:11:25.966089 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.966075 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 08 12:11:25.966319 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.966302 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 08 12:11:25.974985 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.973864 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 08 12:11:25.989342 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:25.989316 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f8649bc-jprlp"] Jun 08 12:11:26.028489 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028448 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-federate-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028489 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028490 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-metrics-client-ca\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028536 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028602 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-serving-certs-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028631 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028648 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vc7q\" (UniqueName: \"kubernetes.io/projected/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-kube-api-access-5vc7q\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.028878 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.028735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129435 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129396 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-federate-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129435 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129435 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-metrics-client-ca\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129466 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129506 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-serving-certs-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129549 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129571 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vc7q\" (UniqueName: \"kubernetes.io/projected/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-kube-api-access-5vc7q\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129606 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.129716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.129656 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.130262 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.130229 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-metrics-client-ca\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.130357 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.130335 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-serving-certs-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.130606 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.130582 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.132146 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.132114 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-federate-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.132276 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.132255 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.132405 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.132388 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.132462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.132445 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-telemeter-client-tls\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.139637 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.139614 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vc7q\" (UniqueName: \"kubernetes.io/projected/b5edfb3a-f9ed-43c1-b92d-07c66bd37912-kube-api-access-5vc7q\") pod \"telemeter-client-9f8649bc-jprlp\" (UID: \"b5edfb3a-f9ed-43c1-b92d-07c66bd37912\") " pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.152487 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.152458 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztpgp" event={"ID":"b1e44096-d501-40a5-bd8a-33e19aa19e20","Type":"ContainerStarted","Data":"4e90a80c438594b80b794f0e829c3de083d11236847ad0b17431023cac108daa"} Jun 08 12:11:26.152809 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.152531 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ztpgp" event={"ID":"b1e44096-d501-40a5-bd8a-33e19aa19e20","Type":"ContainerStarted","Data":"4d6fa51b48b0642c6f0ef5ef89173ac79f87980a5f512934ca10534a8f209f55"} Jun 08 12:11:26.168415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.168368 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ztpgp" podStartSLOduration=253.211516945 podStartE2EDuration="4m14.168352729s" totalImagesPullingTime="956.835784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:26.167306482 +0000 UTC m=+254.277562705" watchObservedRunningTime="2026-06-08 12:11:26.168352729 +0000 UTC m=+254.278608950" Jun 08 12:11:26.276539 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.276434 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" Jun 08 12:11:26.409581 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:26.409544 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f8649bc-jprlp"] Jun 08 12:11:26.413012 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:26.412988 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5edfb3a_f9ed_43c1_b92d_07c66bd37912.slice/crio-d20d2fe41c6623a9cb4b4a9121405be801f0ad2b097ad93225ed09abfa546793 WatchSource:0}: Error finding container d20d2fe41c6623a9cb4b4a9121405be801f0ad2b097ad93225ed09abfa546793: Status 404 returned error can't find the container with id d20d2fe41c6623a9cb4b4a9121405be801f0ad2b097ad93225ed09abfa546793 Jun 08 12:11:27.157459 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:27.157424 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" event={"ID":"b5edfb3a-f9ed-43c1-b92d-07c66bd37912","Type":"ContainerStarted","Data":"d20d2fe41c6623a9cb4b4a9121405be801f0ad2b097ad93225ed09abfa546793"} Jun 08 12:11:29.166917 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:29.166881 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" event={"ID":"b5edfb3a-f9ed-43c1-b92d-07c66bd37912","Type":"ContainerStarted","Data":"d5190633821f2fa98abebcb8b59efb7f59eb05f93cb3c3cda430e43dfc462adc"} Jun 08 12:11:30.172652 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:30.172609 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" event={"ID":"b5edfb3a-f9ed-43c1-b92d-07c66bd37912","Type":"ContainerStarted","Data":"5b0db246a11f8d156501669ce7813a5d14b4e773f6ec9c7ffebd3e9c65df3027"} Jun 08 12:11:30.173053 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:30.172715 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" event={"ID":"b5edfb3a-f9ed-43c1-b92d-07c66bd37912","Type":"ContainerStarted","Data":"939e7269d1ea268a81ce94a0285c21c0f14657849112036cd6b3b46567e7dba9"} Jun 08 12:11:30.198809 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:30.198750 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9f8649bc-jprlp" podStartSLOduration=2.459899535 podStartE2EDuration="5.19873442s" totalImagesPullingTime="2.738834885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:30.196583663 +0000 UTC m=+258.306839885" watchObservedRunningTime="2026-06-08 12:11:30.19873442 +0000 UTC m=+258.308990641" Jun 08 12:11:33.868834 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.868799 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-lp8ct"] Jun 08 12:11:33.871882 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.871862 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:33.875300 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.875276 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mk7lm\"" Jun 08 12:11:33.875300 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.875290 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 08 12:11:33.875471 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.875310 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 08 12:11:33.882083 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.882057 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-lp8ct"] Jun 08 12:11:33.895689 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.895660 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47twm\" (UniqueName: \"kubernetes.io/projected/eeb625a7-05b2-4f76-ae7f-098c104a72c6-kube-api-access-47twm\") pod \"downloads-6657875f97-lp8ct\" (UID: \"eeb625a7-05b2-4f76-ae7f-098c104a72c6\") " pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:33.997075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:33.997035 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47twm\" (UniqueName: \"kubernetes.io/projected/eeb625a7-05b2-4f76-ae7f-098c104a72c6-kube-api-access-47twm\") pod \"downloads-6657875f97-lp8ct\" (UID: \"eeb625a7-05b2-4f76-ae7f-098c104a72c6\") " pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:34.007196 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:34.007168 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47twm\" (UniqueName: \"kubernetes.io/projected/eeb625a7-05b2-4f76-ae7f-098c104a72c6-kube-api-access-47twm\") pod \"downloads-6657875f97-lp8ct\" (UID: \"eeb625a7-05b2-4f76-ae7f-098c104a72c6\") " pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:34.180849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:34.180765 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:34.336495 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:34.336457 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-lp8ct"] Jun 08 12:11:34.339373 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:34.339344 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb625a7_05b2_4f76_ae7f_098c104a72c6.slice/crio-140fd63489bdf0a6ef1788112c928f137d22ac830356fce7dcf9d1dc06e527ac WatchSource:0}: Error finding container 140fd63489bdf0a6ef1788112c928f137d22ac830356fce7dcf9d1dc06e527ac: Status 404 returned error can't find the container with id 140fd63489bdf0a6ef1788112c928f137d22ac830356fce7dcf9d1dc06e527ac Jun 08 12:11:35.193176 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:35.193136 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-lp8ct" event={"ID":"eeb625a7-05b2-4f76-ae7f-098c104a72c6","Type":"ContainerStarted","Data":"140fd63489bdf0a6ef1788112c928f137d22ac830356fce7dcf9d1dc06e527ac"} Jun 08 12:11:44.192328 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.192297 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:11:44.197763 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.197743 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.200838 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.200810 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 08 12:11:44.200980 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.200880 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 08 12:11:44.200980 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.200945 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 08 12:11:44.202005 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.201983 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 08 12:11:44.202121 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.202055 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2jkfq\"" Jun 08 12:11:44.202121 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.202094 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 08 12:11:44.207160 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.207079 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 08 12:11:44.210419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.210396 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:11:44.285285 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285248 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285482 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285293 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4r8d\" (UniqueName: \"kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285482 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285370 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285482 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285393 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285659 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285485 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285699 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.285734 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.285709 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386708 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386744 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4r8d\" (UniqueName: \"kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386799 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386829 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386860 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386916 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.386990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.386957 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.387882 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.387712 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.387882 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.387740 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.387882 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.387828 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.388161 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.388138 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.389750 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.389727 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.389874 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.389810 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.395338 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.395316 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4r8d\" (UniqueName: \"kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d\") pod \"console-56f88559fc-nmgmc\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.510114 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.510029 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:11:44.651899 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:44.651861 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:11:44.656181 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:44.656146 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod614b62ee_5483_40af_8f68_f9b7ff82f266.slice/crio-934db73f1f764be01a74fff878636aebde33e6f0206c8bccaa67285add925028 WatchSource:0}: Error finding container 934db73f1f764be01a74fff878636aebde33e6f0206c8bccaa67285add925028: Status 404 returned error can't find the container with id 934db73f1f764be01a74fff878636aebde33e6f0206c8bccaa67285add925028 Jun 08 12:11:45.236916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:45.236868 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f88559fc-nmgmc" event={"ID":"614b62ee-5483-40af-8f68-f9b7ff82f266","Type":"ContainerStarted","Data":"934db73f1f764be01a74fff878636aebde33e6f0206c8bccaa67285add925028"} Jun 08 12:11:53.865829 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:11:53.865775 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" Jun 08 12:11:53.866272 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:11:53.865775 2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bv5jn" podUID="a9ef63cf-e8c1-427a-8376-18b4dc9d6cce" Jun 08 12:11:54.272120 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:54.272031 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:11:54.272295 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:54.272042 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:11:55.278318 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.278278 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f88559fc-nmgmc" event={"ID":"614b62ee-5483-40af-8f68-f9b7ff82f266","Type":"ContainerStarted","Data":"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4"} Jun 08 12:11:55.280059 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.280029 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-lp8ct" event={"ID":"eeb625a7-05b2-4f76-ae7f-098c104a72c6","Type":"ContainerStarted","Data":"ec23fd06c865aac13910e5c90dec17ac15340e7de2f1b66f55c9bd9d96710389"} Jun 08 12:11:55.280269 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.280241 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:55.296334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.296273 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f88559fc-nmgmc" podStartSLOduration=1.547746879 podStartE2EDuration="11.296252274s" totalImagesPullingTime="9.748505395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:55.294637357 +0000 UTC m=+283.404893578" watchObservedRunningTime="2026-06-08 12:11:55.296252274 +0000 UTC m=+283.406508499" Jun 08 12:11:55.297529 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.297477 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-lp8ct" Jun 08 12:11:55.310171 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:55.310119 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-lp8ct" podStartSLOduration=2.213462671 podStartE2EDuration="22.310105185s" totalImagesPullingTime="20.096642514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:55.309410006 +0000 UTC m=+283.419666221" watchObservedRunningTime="2026-06-08 12:11:55.310105185 +0000 UTC m=+283.420361407" Jun 08 12:11:57.674177 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.674128 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" podUID="a8f7fab3-e91b-4775-95cb-52967acbd1b2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 12:11:57.705452 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.705410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:11:57.705647 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.705536 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:11:57.708194 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.708168 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a9ef63cf-e8c1-427a-8376-18b4dc9d6cce-metrics-tls\") pod \"dns-default-bv5jn\" (UID: \"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce\") " pod="openshift-dns/dns-default-bv5jn" Jun 08 12:11:57.708326 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.708307 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"image-registry-667ff6457b-c86qv\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:11:57.876841 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.876807 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jwnkf\"" Jun 08 12:11:57.877076 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.876809 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6km9q\"" Jun 08 12:11:57.883046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.883016 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:11:57.883198 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:57.883128 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:11:58.061043 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.060959 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bv5jn"] Jun 08 12:11:58.064862 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:58.064828 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ef63cf_e8c1_427a_8376_18b4dc9d6cce.slice/crio-7750d01aae6830f7be099e5d4d4796450a57072b4ada11d5484d6e3a500522aa WatchSource:0}: Error finding container 7750d01aae6830f7be099e5d4d4796450a57072b4ada11d5484d6e3a500522aa: Status 404 returned error can't find the container with id 7750d01aae6830f7be099e5d4d4796450a57072b4ada11d5484d6e3a500522aa Jun 08 12:11:58.108489 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.108450 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:11:58.112218 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:11:58.112185 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7f1cbb1_67f7_4eec_97a1_47bf4388a0eb.slice/crio-8d2fd39ec142251be9eb0102c518259df7c7ef882d49c2d70783a75523c36028 WatchSource:0}: Error finding container 8d2fd39ec142251be9eb0102c518259df7c7ef882d49c2d70783a75523c36028: Status 404 returned error can't find the container with id 8d2fd39ec142251be9eb0102c518259df7c7ef882d49c2d70783a75523c36028 Jun 08 12:11:58.293736 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.293693 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" event={"ID":"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb","Type":"ContainerStarted","Data":"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53"} Jun 08 12:11:58.293923 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.293760 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" event={"ID":"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb","Type":"ContainerStarted","Data":"8d2fd39ec142251be9eb0102c518259df7c7ef882d49c2d70783a75523c36028"} Jun 08 12:11:58.293923 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.293786 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:11:58.294985 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.294949 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5jn" event={"ID":"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce","Type":"ContainerStarted","Data":"7750d01aae6830f7be099e5d4d4796450a57072b4ada11d5484d6e3a500522aa"} Jun 08 12:11:58.321461 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:11:58.321410 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" podStartSLOduration=285.321394435 podStartE2EDuration="4m45.321394435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:11:58.320344999 +0000 UTC m=+286.430601222" watchObservedRunningTime="2026-06-08 12:11:58.321394435 +0000 UTC m=+286.431650656" Jun 08 12:12:01.311053 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:01.311002 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5jn" event={"ID":"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce","Type":"ContainerStarted","Data":"d3de964d89ec83b1946e547aa10c68b968f9252da63832f60da01c169bbdabeb"} Jun 08 12:12:01.311622 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:01.311063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bv5jn" event={"ID":"a9ef63cf-e8c1-427a-8376-18b4dc9d6cce","Type":"ContainerStarted","Data":"9bdcc17dffe64893157e8b4a7764754d1b3e68bcbfaa96bb6cb5be8af6bf30cf"} Jun 08 12:12:01.311622 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:01.311098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:12:01.333004 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:01.332943 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bv5jn" podStartSLOduration=252.067300065 podStartE2EDuration="4m14.332926102s" totalImagesPullingTime="2.265626037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:07:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:12:01.331112454 +0000 UTC m=+289.441368677" watchObservedRunningTime="2026-06-08 12:12:01.332926102 +0000 UTC m=+289.443182325" Jun 08 12:12:04.511110 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:04.511066 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:12:04.511640 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:04.511131 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:12:04.512428 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:04.512408 2576 patch_prober.go:28] interesting pod/console-56f88559fc-nmgmc container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body= Jun 08 12:12:04.512479 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:04.512449 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-56f88559fc-nmgmc" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" probeResult="failure" output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" Jun 08 12:12:07.673714 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:07.673677 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" podUID="a8f7fab3-e91b-4775-95cb-52967acbd1b2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 12:12:11.317358 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:11.317328 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bv5jn" Jun 08 12:12:12.298422 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:12.298398 2576 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 12:12:14.511275 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:14.511234 2576 patch_prober.go:28] interesting pod/console-56f88559fc-nmgmc container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" start-of-body= Jun 08 12:12:14.532583 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:14.511302 2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-56f88559fc-nmgmc" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" probeResult="failure" output="Get \"https://10.132.0.15:8443/health\": dial tcp 10.132.0.15:8443: connect: connection refused" Jun 08 12:12:17.673944 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.673904 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" podUID="a8f7fab3-e91b-4775-95cb-52967acbd1b2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 08 12:12:17.674398 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.673981 2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" Jun 08 12:12:17.674450 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.674410 2576 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c916dfc45ca57cc7b47e7d96e02a3df3fa2793ac47d847a2cbc1a2beae3bc220"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 08 12:12:17.674497 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.674454 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" podUID="a8f7fab3-e91b-4775-95cb-52967acbd1b2" containerName="service-proxy" containerID="cri-o://c916dfc45ca57cc7b47e7d96e02a3df3fa2793ac47d847a2cbc1a2beae3bc220" gracePeriod=30 Jun 08 12:12:17.887371 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.887341 2576 patch_prober.go:28] interesting pod/image-registry-667ff6457b-c86qv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 12:12:17.887559 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:17.887410 2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 12:12:18.371732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:18.371696 2576 generic.go:356] "Generic (PLEG): container finished" podID="a8f7fab3-e91b-4775-95cb-52967acbd1b2" containerID="c916dfc45ca57cc7b47e7d96e02a3df3fa2793ac47d847a2cbc1a2beae3bc220" exitCode=2 Jun 08 12:12:18.371899 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:18.371762 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerDied","Data":"c916dfc45ca57cc7b47e7d96e02a3df3fa2793ac47d847a2cbc1a2beae3bc220"} Jun 08 12:12:18.371899 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:18.371803 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-b66cb8d-9fbjl" event={"ID":"a8f7fab3-e91b-4775-95cb-52967acbd1b2","Type":"ContainerStarted","Data":"137072cae031f96493f90f40c1ea38d5a84da3deed51952e26f6d5f200195971"} Jun 08 12:12:19.304128 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:19.304098 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:12:24.515882 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:24.515850 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:12:24.519794 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:24.519770 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:12:37.989161 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:12:37.989122 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:13:03.009027 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.008923 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerName="registry" containerID="cri-o://e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53" gracePeriod=30 Jun 08 12:13:03.242215 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.242192 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:13:03.287652 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287566 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287652 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287625 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287671 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287695 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287715 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287736 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287772 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.287857 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.287800 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9lq4\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4\") pod \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\" (UID: \"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb\") " Jun 08 12:13:03.288183 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.288149 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:03.288246 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.288178 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:03.290104 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.290053 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:13:03.290232 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.290216 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4" (OuterVolumeSpecName: "kube-api-access-t9lq4") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "kube-api-access-t9lq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:13:03.290349 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.290328 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:13:03.290473 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.290454 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:13:03.290557 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.290511 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:13:03.297102 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.297080 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" (UID: "a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:13:03.388806 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388771 2576 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-ca-trust-extracted\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.388806 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388803 2576 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-bound-sa-token\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.388806 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388813 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-trusted-ca\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.389019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388820 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.389019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388830 2576 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-registry-certificates\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.389019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388838 2576 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-installation-pull-secrets\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.389019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388846 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9lq4\" (UniqueName: \"kubernetes.io/projected/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-kube-api-access-t9lq4\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.389019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.388854 2576 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb-image-registry-private-configuration\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:03.525923 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.525894 2576 generic.go:356] "Generic (PLEG): container finished" podID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerID="e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53" exitCode=0 Jun 08 12:13:03.526080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.525941 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" Jun 08 12:13:03.526080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.525975 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" event={"ID":"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb","Type":"ContainerDied","Data":"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53"} Jun 08 12:13:03.526080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.526012 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-667ff6457b-c86qv" event={"ID":"a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb","Type":"ContainerDied","Data":"8d2fd39ec142251be9eb0102c518259df7c7ef882d49c2d70783a75523c36028"} Jun 08 12:13:03.526080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.526031 2576 scope.go:117] "RemoveContainer" containerID="e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53" Jun 08 12:13:03.534758 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.534737 2576 scope.go:117] "RemoveContainer" containerID="e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53" Jun 08 12:13:03.534997 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:13:03.534977 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53\": container with ID starting with e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53 not found: ID does not exist" containerID="e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53" Jun 08 12:13:03.535063 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.535008 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53"} err="failed to get container status \"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53\": rpc error: code = NotFound desc = could not find container \"e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53\": container with ID starting with e328b5db4938352517837596fff5d22ab4564f6062ded12060b6da425106fd53 not found: ID does not exist" Jun 08 12:13:03.546942 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.546917 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:13:03.552375 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:03.552351 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-667ff6457b-c86qv"] Jun 08 12:13:04.346046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:04.346009 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" path="/var/lib/kubelet/pods/a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb/volumes" Jun 08 12:13:05.356108 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.356076 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:13:05.356457 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.356304 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerName="registry" Jun 08 12:13:05.356457 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.356314 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerName="registry" Jun 08 12:13:05.356457 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.356367 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7f1cbb1-67f7-4eec-97a1-47bf4388a0eb" containerName="registry" Jun 08 12:13:05.359423 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.359408 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.371096 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.371075 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:13:05.402952 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.402910 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403090 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.402988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh2d9\" (UniqueName: \"kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403090 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.403023 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403090 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.403062 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403090 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.403089 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403283 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.403137 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.403283 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.403175 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503610 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503580 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh2d9\" (UniqueName: \"kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503610 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503613 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503634 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503667 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503687 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503711 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.503825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.503736 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.504437 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.504408 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.504556 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.504532 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.504597 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.504547 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.504658 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.504641 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.506680 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.506656 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.506787 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.506767 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.513940 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.513908 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh2d9\" (UniqueName: \"kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9\") pod \"console-7bdff67cbf-plkhs\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.667582 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.667486 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:05.796384 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:05.796346 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:13:05.799085 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:13:05.799057 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcad4e60_914c_43b8_9b9e_65dcbbb7f293.slice/crio-754a8921cb6000d6e7cc4d16952e73fd6c3da9a5d3014763bca2b25289db61b3 WatchSource:0}: Error finding container 754a8921cb6000d6e7cc4d16952e73fd6c3da9a5d3014763bca2b25289db61b3: Status 404 returned error can't find the container with id 754a8921cb6000d6e7cc4d16952e73fd6c3da9a5d3014763bca2b25289db61b3 Jun 08 12:13:06.538074 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:06.538033 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdff67cbf-plkhs" event={"ID":"fcad4e60-914c-43b8-9b9e-65dcbbb7f293","Type":"ContainerStarted","Data":"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f"} Jun 08 12:13:06.538445 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:06.538097 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdff67cbf-plkhs" event={"ID":"fcad4e60-914c-43b8-9b9e-65dcbbb7f293","Type":"ContainerStarted","Data":"754a8921cb6000d6e7cc4d16952e73fd6c3da9a5d3014763bca2b25289db61b3"} Jun 08 12:13:06.566087 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:06.566034 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bdff67cbf-plkhs" podStartSLOduration=1.566014426 podStartE2EDuration="1.566014426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:13:06.564656433 +0000 UTC m=+354.674912655" watchObservedRunningTime="2026-06-08 12:13:06.566014426 +0000 UTC m=+354.676270650" Jun 08 12:13:15.668378 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:15.668343 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:15.668795 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:15.668411 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:15.672974 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:15.672953 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:16.578303 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:16.578276 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:13:16.634153 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:16.634115 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:13:41.654935 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.654884 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56f88559fc-nmgmc" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" containerID="cri-o://7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4" gracePeriod=15 Jun 08 12:13:41.898863 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.898838 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:13:41.992110 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992027 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992110 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992089 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4r8d\" (UniqueName: \"kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992110 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992110 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992362 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992149 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992362 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992176 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992362 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992202 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992362 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992226 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config\") pod \"614b62ee-5483-40af-8f68-f9b7ff82f266\" (UID: \"614b62ee-5483-40af-8f68-f9b7ff82f266\") " Jun 08 12:13:41.992362 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992326 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca" (OuterVolumeSpecName: "service-ca") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:41.992716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992455 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-service-ca\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:41.992716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992562 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:41.992826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992718 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config" (OuterVolumeSpecName: "console-config") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:41.992826 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.992754 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:13:41.994426 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.994395 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:13:41.994551 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.994427 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d" (OuterVolumeSpecName: "kube-api-access-w4r8d") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "kube-api-access-w4r8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:13:41.994551 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:41.994428 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "614b62ee-5483-40af-8f68-f9b7ff82f266" (UID: "614b62ee-5483-40af-8f68-f9b7ff82f266"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:13:42.093006 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.092948 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-oauth-config\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.093006 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.093001 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4r8d\" (UniqueName: \"kubernetes.io/projected/614b62ee-5483-40af-8f68-f9b7ff82f266-kube-api-access-w4r8d\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.093006 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.093012 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-trusted-ca-bundle\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.093006 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.093021 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-oauth-serving-cert\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.093266 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.093029 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/614b62ee-5483-40af-8f68-f9b7ff82f266-console-serving-cert\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.093266 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.093038 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/614b62ee-5483-40af-8f68-f9b7ff82f266-console-config\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:13:42.671311 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.671218 2576 generic.go:356] "Generic (PLEG): container finished" podID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerID="7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4" exitCode=0 Jun 08 12:13:42.671311 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.671260 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f88559fc-nmgmc" Jun 08 12:13:42.671311 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.671270 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f88559fc-nmgmc" event={"ID":"614b62ee-5483-40af-8f68-f9b7ff82f266","Type":"ContainerDied","Data":"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4"} Jun 08 12:13:42.671878 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.671316 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f88559fc-nmgmc" event={"ID":"614b62ee-5483-40af-8f68-f9b7ff82f266","Type":"ContainerDied","Data":"934db73f1f764be01a74fff878636aebde33e6f0206c8bccaa67285add925028"} Jun 08 12:13:42.671878 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.671339 2576 scope.go:117] "RemoveContainer" containerID="7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4" Jun 08 12:13:42.679231 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.679215 2576 scope.go:117] "RemoveContainer" containerID="7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4" Jun 08 12:13:42.679507 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:13:42.679489 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4\": container with ID starting with 7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4 not found: ID does not exist" containerID="7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4" Jun 08 12:13:42.679588 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.679538 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4"} err="failed to get container status \"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4\": rpc error: code = NotFound desc = could not find container \"7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4\": container with ID starting with 7c5773543d20c3166a03a9dec5478da36ad97c1ec539f94dc29e9fb63b1204b4 not found: ID does not exist" Jun 08 12:13:42.688113 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.688090 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:13:42.692431 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:42.692411 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f88559fc-nmgmc"] Jun 08 12:13:44.345652 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:13:44.345620 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" path="/var/lib/kubelet/pods/614b62ee-5483-40af-8f68-f9b7ff82f266/volumes" Jun 08 12:15:41.692935 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.692895 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:15:41.693421 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.693232 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" Jun 08 12:15:41.693421 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.693246 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" Jun 08 12:15:41.693421 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.693296 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="614b62ee-5483-40af-8f68-f9b7ff82f266" containerName="console" Jun 08 12:15:41.694862 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.694848 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:41.697433 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.697405 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jun 08 12:15:41.697596 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.697444 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-nwd6c\"" Jun 08 12:15:41.698470 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.698452 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jun 08 12:15:41.710400 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.710376 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:15:41.762580 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.762541 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt9m9\" (UniqueName: \"kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9\") pod \"feast-operator-controller-manager-78576d7dd5-482xb\" (UID: \"8290a8ca-37af-4eca-9006-3a50fbc17a06\") " pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:41.862927 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.862891 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt9m9\" (UniqueName: \"kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9\") pod \"feast-operator-controller-manager-78576d7dd5-482xb\" (UID: \"8290a8ca-37af-4eca-9006-3a50fbc17a06\") " pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:41.871187 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:41.871155 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt9m9\" (UniqueName: \"kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9\") pod \"feast-operator-controller-manager-78576d7dd5-482xb\" (UID: \"8290a8ca-37af-4eca-9006-3a50fbc17a06\") " pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:42.004929 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:42.004833 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:42.131728 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:42.131703 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:15:42.134067 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:15:42.134037 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8290a8ca_37af_4eca_9006_3a50fbc17a06.slice/crio-3065aea0409e5a5dd469b623ad79954fc02b94eedc78d729bca0e134080f7759 WatchSource:0}: Error finding container 3065aea0409e5a5dd469b623ad79954fc02b94eedc78d729bca0e134080f7759: Status 404 returned error can't find the container with id 3065aea0409e5a5dd469b623ad79954fc02b94eedc78d729bca0e134080f7759 Jun 08 12:15:42.136644 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:42.136629 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:15:43.086933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:43.086888 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" event={"ID":"8290a8ca-37af-4eca-9006-3a50fbc17a06","Type":"ContainerStarted","Data":"3065aea0409e5a5dd469b623ad79954fc02b94eedc78d729bca0e134080f7759"} Jun 08 12:15:45.096411 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:45.096375 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" event={"ID":"8290a8ca-37af-4eca-9006-3a50fbc17a06","Type":"ContainerStarted","Data":"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773"} Jun 08 12:15:45.096896 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:45.096474 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:15:45.111213 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:45.111158 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" podStartSLOduration=1.306355262 podStartE2EDuration="4.111141975s" totalImagesPullingTime="2.804786713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:15:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:15:45.11060124 +0000 UTC m=+513.220857463" watchObservedRunningTime="2026-06-08 12:15:45.111141975 +0000 UTC m=+513.221398197" Jun 08 12:15:56.102903 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:15:56.102874 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:16:25.183749 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.183657 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:16:25.188713 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.188688 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.193342 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193317 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193346 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193375 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193417 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193381 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193469 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193419 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jun 08 12:16:25.193512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.193489 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7rxcf\"" Jun 08 12:16:25.205355 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.205329 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:16:25.264778 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264735 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbb25\" (UniqueName: \"kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.264778 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264775 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.265046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264795 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.265046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264823 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.265046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264875 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.265046 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.264965 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366180 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366138 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbb25\" (UniqueName: \"kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366180 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366182 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366358 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366391 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366602 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:16:25.366362 2576 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jun 08 12:16:25.366602 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366472 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.366602 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:16:25.366552 2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls podName:42ebc334-2944-4e17-9659-701b011eb765 nodeName:}" failed. No retries permitted until 2026-06-08 12:16:25.866497135 +0000 UTC m=+553.976753349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls") pod "feast-simple-feast-setup-c55dcf4bd-57wsz" (UID: "42ebc334-2944-4e17-9659-701b011eb765") : secret "feast-simple-feast-setup-registry-tls" not found Jun 08 12:16:25.366804 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.366781 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.368885 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.368863 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.369075 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.369055 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.369141 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.369065 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.381415 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.381385 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbb25\" (UniqueName: \"kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.870443 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.870402 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:25.872954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:25.872922 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-57wsz\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:26.097723 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:26.097682 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:16:26.243840 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:26.243815 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:16:26.245096 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:16:26.245074 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42ebc334_2944_4e17_9659_701b011eb765.slice/crio-a9ae41e2d0438f250bec06aa5a57d3ab16fd6f3f1655cd80f3c256eb1ce94299 WatchSource:0}: Error finding container a9ae41e2d0438f250bec06aa5a57d3ab16fd6f3f1655cd80f3c256eb1ce94299: Status 404 returned error can't find the container with id a9ae41e2d0438f250bec06aa5a57d3ab16fd6f3f1655cd80f3c256eb1ce94299 Jun 08 12:16:27.248033 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:16:27.247959 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"a9ae41e2d0438f250bec06aa5a57d3ab16fd6f3f1655cd80f3c256eb1ce94299"} Jun 08 12:17:00.374087 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:00.374053 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"6b88c385c70f261089884d1085d58abf5839306400c5f4a943fbd3a0f8cab1c2"} Jun 08 12:17:04.391808 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:04.391772 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="6b88c385c70f261089884d1085d58abf5839306400c5f4a943fbd3a0f8cab1c2" exitCode=0 Jun 08 12:17:04.392289 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:04.391846 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"6b88c385c70f261089884d1085d58abf5839306400c5f4a943fbd3a0f8cab1c2"} Jun 08 12:17:05.397880 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:05.397849 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"36b8568feecd52b2e151fd8c81e2b61cbaa0e283a3eeb8caff5d1138a45d8010"} Jun 08 12:17:09.147901 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:09.147861 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:17:10.416676 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:10.416642 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="36b8568feecd52b2e151fd8c81e2b61cbaa0e283a3eeb8caff5d1138a45d8010" exitCode=0 Jun 08 12:17:10.417055 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:10.416682 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"36b8568feecd52b2e151fd8c81e2b61cbaa0e283a3eeb8caff5d1138a45d8010"} Jun 08 12:17:11.426849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:11.426806 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"73ab6fd881d1fbdfc70901b4ec832bbe0d6c4e45adc42f4fda002f816e113439"} Jun 08 12:17:11.427307 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:11.426874 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"22e44806ec779dc54dc2ff3a01753d8a391ec87462984231af9c283beb8b0aaa"} Jun 08 12:17:11.427307 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:11.426897 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"f278f1599697b778b01f65f3687860f249b4f7043c3b12e821bc675e2b3cda7d"} Jun 08 12:17:11.427307 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:11.426917 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerStarted","Data":"05b57cc27b3ddcff3a953d6cb397bfaf1780db2790012ad1e148ccd2fe10baec"} Jun 08 12:17:11.454313 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:11.454249 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podStartSLOduration=13.325585512 podStartE2EDuration="46.454225315s" totalImagesPullingTime="33.128639803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:17:11.45305876 +0000 UTC m=+599.563314983" watchObservedRunningTime="2026-06-08 12:17:11.454225315 +0000 UTC m=+599.564481540" Jun 08 12:17:14.099272 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.098456 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:14.099272 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.098597 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:14.099272 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.098636 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:14.099272 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.099154 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:14.101409 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.101368 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Jun 08 12:17:14.101568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.101442 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.18:8816: connect: connection refused" Jun 08 12:17:14.101568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.101439 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Jun 08 12:17:14.101749 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:14.101720 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.18:6571: connect: connection refused" Jun 08 12:17:16.098308 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:16.098264 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:16.100959 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:16.098335 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:16.100959 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:16.098357 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:16.100959 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:16.098375 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.099533 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.099493 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.099961 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.099782 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.100026 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.100004 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.100329 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.100306 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.100455 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.100375 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.100498 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.100481 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.104054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.104038 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:17.457565 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:17.457478 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:17:34.167485 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.167388 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7bdff67cbf-plkhs" podUID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" containerName="console" containerID="cri-o://5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f" gracePeriod=15 Jun 08 12:17:34.404578 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.404553 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:17:34.518966 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.518875 2576 generic.go:356] "Generic (PLEG): container finished" podID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" containerID="5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f" exitCode=0 Jun 08 12:17:34.518966 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.518926 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bdff67cbf-plkhs" Jun 08 12:17:34.519148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.518963 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdff67cbf-plkhs" event={"ID":"fcad4e60-914c-43b8-9b9e-65dcbbb7f293","Type":"ContainerDied","Data":"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f"} Jun 08 12:17:34.519148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.519013 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bdff67cbf-plkhs" event={"ID":"fcad4e60-914c-43b8-9b9e-65dcbbb7f293","Type":"ContainerDied","Data":"754a8921cb6000d6e7cc4d16952e73fd6c3da9a5d3014763bca2b25289db61b3"} Jun 08 12:17:34.519148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.519034 2576 scope.go:117] "RemoveContainer" containerID="5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f" Jun 08 12:17:34.526689 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.526674 2576 scope.go:117] "RemoveContainer" containerID="5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f" Jun 08 12:17:34.526977 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:17:34.526956 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f\": container with ID starting with 5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f not found: ID does not exist" containerID="5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f" Jun 08 12:17:34.527048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.526986 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f"} err="failed to get container status \"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f\": rpc error: code = NotFound desc = could not find container \"5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f\": container with ID starting with 5b81d4af4f2e15cafb3d54be93a3e8ffb0159c64b71b10526afead48d59a560f not found: ID does not exist" Jun 08 12:17:34.555258 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555230 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh2d9\" (UniqueName: \"kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555370 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555277 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555370 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555307 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555370 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555333 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555494 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555368 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555494 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555415 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555494 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555444 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca\") pod \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\" (UID: \"fcad4e60-914c-43b8-9b9e-65dcbbb7f293\") " Jun 08 12:17:34.555927 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555760 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config" (OuterVolumeSpecName: "console-config") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:17:34.555927 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555789 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:17:34.555927 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.555810 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:17:34.556102 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.556005 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca" (OuterVolumeSpecName: "service-ca") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:17:34.557429 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.557401 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:17:34.557555 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.557434 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:17:34.557555 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.557456 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9" (OuterVolumeSpecName: "kube-api-access-xh2d9") pod "fcad4e60-914c-43b8-9b9e-65dcbbb7f293" (UID: "fcad4e60-914c-43b8-9b9e-65dcbbb7f293"). InnerVolumeSpecName "kube-api-access-xh2d9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:17:34.656474 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656447 2576 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-serving-cert\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656474 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656471 2576 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-service-ca\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656474 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656481 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh2d9\" (UniqueName: \"kubernetes.io/projected/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-kube-api-access-xh2d9\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656715 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656490 2576 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-config\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656715 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656499 2576 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-oauth-serving-cert\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656715 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656506 2576 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-console-oauth-config\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.656715 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.656532 2576 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fcad4e60-914c-43b8-9b9e-65dcbbb7f293-trusted-ca-bundle\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:17:34.845812 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.845784 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:17:34.854162 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:34.854132 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bdff67cbf-plkhs"] Jun 08 12:17:36.345454 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:17:36.345418 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" path="/var/lib/kubelet/pods/fcad4e60-914c-43b8-9b9e-65dcbbb7f293/volumes" Jun 08 12:18:20.679498 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:20.679463 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:18:20.680066 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:20.679791 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="registry" containerID="cri-o://05b57cc27b3ddcff3a953d6cb397bfaf1780db2790012ad1e148ccd2fe10baec" gracePeriod=30 Jun 08 12:18:20.680066 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:20.679849 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="offline" containerID="cri-o://22e44806ec779dc54dc2ff3a01753d8a391ec87462984231af9c283beb8b0aaa" gracePeriod=30 Jun 08 12:18:20.680066 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:20.679883 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" containerID="cri-o://73ab6fd881d1fbdfc70901b4ec832bbe0d6c4e45adc42f4fda002f816e113439" gracePeriod=30 Jun 08 12:18:20.680234 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:20.679913 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" containerID="cri-o://f278f1599697b778b01f65f3687860f249b4f7043c3b12e821bc675e2b3cda7d" gracePeriod=30 Jun 08 12:18:21.088001 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.087963 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:18:21.088372 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.088355 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" containerName="console" Jun 08 12:18:21.088461 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.088377 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" containerName="console" Jun 08 12:18:21.088496 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.088464 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcad4e60-914c-43b8-9b9e-65dcbbb7f293" containerName="console" Jun 08 12:18:21.091841 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.091821 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.097117 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.097098 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-7xs78\"" Jun 08 12:18:21.203052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203022 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.203195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203057 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.203195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203072 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.203195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203114 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.203195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203153 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.203195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.203169 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skwsv\" (UniqueName: \"kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.218266 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.218235 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:18:21.304225 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304180 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304235 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304259 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304301 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304362 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304419 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304383 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skwsv\" (UniqueName: \"kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.304822 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.304794 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.307411 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.307378 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.307503 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.307431 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.307503 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.307447 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.307619 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.307581 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.320799 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.320778 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skwsv\" (UniqueName: \"kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv\") pod \"feast-simple-feast-setup-c55dcf4bd-lh2nz\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.402745 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.402726 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:21.573305 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:18:21.573270 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb84736a_ca8f_4b43_aad0_c1b145deeab8.slice/crio-23d8ac193516742fedfbcf771c9ade0cbeac7b6c76718289c63d8460b084919d WatchSource:0}: Error finding container 23d8ac193516742fedfbcf771c9ade0cbeac7b6c76718289c63d8460b084919d: Status 404 returned error can't find the container with id 23d8ac193516742fedfbcf771c9ade0cbeac7b6c76718289c63d8460b084919d Jun 08 12:18:21.578256 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.578227 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:18:21.690054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.689974 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"7f617abc13cb3901fba4e927c68b34e4b76190064c4443bc11c6213508ccf39b"} Jun 08 12:18:21.690054 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.690025 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"23d8ac193516742fedfbcf771c9ade0cbeac7b6c76718289c63d8460b084919d"} Jun 08 12:18:21.693169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.693145 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="73ab6fd881d1fbdfc70901b4ec832bbe0d6c4e45adc42f4fda002f816e113439" exitCode=0 Jun 08 12:18:21.693169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.693165 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="f278f1599697b778b01f65f3687860f249b4f7043c3b12e821bc675e2b3cda7d" exitCode=0 Jun 08 12:18:21.693312 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.693219 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"73ab6fd881d1fbdfc70901b4ec832bbe0d6c4e45adc42f4fda002f816e113439"} Jun 08 12:18:21.693312 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:21.693266 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"f278f1599697b778b01f65f3687860f249b4f7043c3b12e821bc675e2b3cda7d"} Jun 08 12:18:26.712851 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:26.712815 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="7f617abc13cb3901fba4e927c68b34e4b76190064c4443bc11c6213508ccf39b" exitCode=0 Jun 08 12:18:26.713294 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:26.712887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"7f617abc13cb3901fba4e927c68b34e4b76190064c4443bc11c6213508ccf39b"} Jun 08 12:18:27.100007 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:27.099969 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Jun 08 12:18:27.454977 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:27.454887 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Jun 08 12:18:27.718740 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:27.718656 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"b047be27e1c0ff4bd126382928f62ff2738a48bc284cf0f00dfa26dec1725b66"} Jun 08 12:18:32.739109 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:32.739077 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="b047be27e1c0ff4bd126382928f62ff2738a48bc284cf0f00dfa26dec1725b66" exitCode=0 Jun 08 12:18:32.739499 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:32.739154 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"b047be27e1c0ff4bd126382928f62ff2738a48bc284cf0f00dfa26dec1725b66"} Jun 08 12:18:33.748217 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:33.748180 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"749b05ff03b2590c9451f350ee7e2047801c91cb188d8eb48497f0390547df6d"} Jun 08 12:18:33.748824 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:33.748799 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"de6fd2902cc7903dbc5d5238ec8b7c9aa896aa42b3b466d0ea6767281100de4d"} Jun 08 12:18:33.749030 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:33.749010 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"d0c3a2d9a8a174f06f033a467c9d88881daa0493ec6147e20fedbe8d952ad858"} Jun 08 12:18:33.749395 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:33.749372 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerStarted","Data":"8e32bdb95ea9e92976440f3e0d790f13510568af9494247ea35d0f7ffee40bd0"} Jun 08 12:18:33.794423 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:33.794349 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podStartSLOduration=12.794326226 podStartE2EDuration="12.794326226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:18:33.793381915 +0000 UTC m=+681.903638158" watchObservedRunningTime="2026-06-08 12:18:33.794326226 +0000 UTC m=+681.904582451" Jun 08 12:18:36.404041 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.403993 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:36.404573 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.404082 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:36.404573 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.404103 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:36.404573 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.404139 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:36.406305 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.406252 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Jun 08 12:18:36.406427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.406306 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Jun 08 12:18:36.406427 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.406278 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.19:8816: connect: connection refused" Jun 08 12:18:36.406675 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:36.406637 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.19:6571: connect: connection refused" Jun 08 12:18:37.100365 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:37.100291 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Jun 08 12:18:37.454922 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:37.454809 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Jun 08 12:18:39.404601 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.404550 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405077 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.404739 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405135 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405118 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405178 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405158 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405230 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405180 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405230 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405200 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405460 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405439 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405605 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405503 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.405693 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.405617 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.409213 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.409197 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.776757 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.776677 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:39.779702 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:39.779683 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:18:47.100954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:47.100887 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.18:8443: connect: connection refused" Jun 08 12:18:47.101573 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:47.101078 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:18:47.455468 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:47.455368 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" probeResult="failure" output="Get \"https://10.132.0.18:6567/health\": dial tcp 10.132.0.18:6567: connect: connection refused" Jun 08 12:18:47.455685 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:47.455500 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:18:50.820463 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:50.820432 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="22e44806ec779dc54dc2ff3a01753d8a391ec87462984231af9c283beb8b0aaa" exitCode=137 Jun 08 12:18:50.820463 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:50.820456 2576 generic.go:356] "Generic (PLEG): container finished" podID="42ebc334-2944-4e17-9659-701b011eb765" containerID="05b57cc27b3ddcff3a953d6cb397bfaf1780db2790012ad1e148ccd2fe10baec" exitCode=137 Jun 08 12:18:50.820872 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:50.820503 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"22e44806ec779dc54dc2ff3a01753d8a391ec87462984231af9c283beb8b0aaa"} Jun 08 12:18:50.820872 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:50.820558 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"05b57cc27b3ddcff3a953d6cb397bfaf1780db2790012ad1e148ccd2fe10baec"} Jun 08 12:18:51.333169 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.333146 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:18:51.449212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449107 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449174 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449195 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449500 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449304 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449500 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449334 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449500 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449375 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbb25\" (UniqueName: \"kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25\") pod \"42ebc334-2944-4e17-9659-701b011eb765\" (UID: \"42ebc334-2944-4e17-9659-701b011eb765\") " Jun 08 12:18:51.449951 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.449906 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data" (OuterVolumeSpecName: "feast-data") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:18:51.451747 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.451708 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls" (OuterVolumeSpecName: "online-tls") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:18:51.451747 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.451737 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:18:51.451877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.451786 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:18:51.451877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.451809 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:18:51.451877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.451807 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25" (OuterVolumeSpecName: "kube-api-access-wbb25") pod "42ebc334-2944-4e17-9659-701b011eb765" (UID: "42ebc334-2944-4e17-9659-701b011eb765"). InnerVolumeSpecName "kube-api-access-wbb25". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:18:51.550612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550555 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbb25\" (UniqueName: \"kubernetes.io/projected/42ebc334-2944-4e17-9659-701b011eb765-kube-api-access-wbb25\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.550612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550604 2576 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-offline-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.550612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550615 2576 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-ui-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.550612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550622 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.550612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550631 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/42ebc334-2944-4e17-9659-701b011eb765-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.550952 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.550638 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/42ebc334-2944-4e17-9659-701b011eb765-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:18:51.826863 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.826827 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" event={"ID":"42ebc334-2944-4e17-9659-701b011eb765","Type":"ContainerDied","Data":"a9ae41e2d0438f250bec06aa5a57d3ab16fd6f3f1655cd80f3c256eb1ce94299"} Jun 08 12:18:51.827273 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.826891 2576 scope.go:117] "RemoveContainer" containerID="73ab6fd881d1fbdfc70901b4ec832bbe0d6c4e45adc42f4fda002f816e113439" Jun 08 12:18:51.827273 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.826897 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz" Jun 08 12:18:51.837772 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.837544 2576 scope.go:117] "RemoveContainer" containerID="22e44806ec779dc54dc2ff3a01753d8a391ec87462984231af9c283beb8b0aaa" Jun 08 12:18:51.849703 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.849679 2576 scope.go:117] "RemoveContainer" containerID="f278f1599697b778b01f65f3687860f249b4f7043c3b12e821bc675e2b3cda7d" Jun 08 12:18:51.857166 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.857148 2576 scope.go:117] "RemoveContainer" containerID="05b57cc27b3ddcff3a953d6cb397bfaf1780db2790012ad1e148ccd2fe10baec" Jun 08 12:18:51.863495 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.863469 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:18:51.864601 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.864583 2576 scope.go:117] "RemoveContainer" containerID="36b8568feecd52b2e151fd8c81e2b61cbaa0e283a3eeb8caff5d1138a45d8010" Jun 08 12:18:51.871681 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.871662 2576 scope.go:117] "RemoveContainer" containerID="6b88c385c70f261089884d1085d58abf5839306400c5f4a943fbd3a0f8cab1c2" Jun 08 12:18:51.888158 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:51.888134 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-57wsz"] Jun 08 12:18:52.345700 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:18:52.345662 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42ebc334-2944-4e17-9659-701b011eb765" path="/var/lib/kubelet/pods/42ebc334-2944-4e17-9659-701b011eb765/volumes" Jun 08 12:19:42.855324 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855230 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855541 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="feast-apply" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855553 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="feast-apply" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855567 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="registry" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855577 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="registry" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855591 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855597 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855603 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="offline" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855608 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="offline" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855615 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855621 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855630 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="feast-init" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855635 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="feast-init" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855687 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="online" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855694 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="offline" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855700 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="ui" Jun 08 12:19:42.855849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.855706 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ebc334-2944-4e17-9659-701b011eb765" containerName="registry" Jun 08 12:19:42.859016 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.858998 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.900184 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900154 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jun 08 12:19:42.900463 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900440 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rb5mg\"" Jun 08 12:19:42.900463 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900452 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jun 08 12:19:42.900611 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900573 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jun 08 12:19:42.900676 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900663 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jun 08 12:19:42.900787 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.900721 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jun 08 12:19:42.903306 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.903290 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jun 08 12:19:42.906689 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.906675 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jun 08 12:19:42.927666 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927642 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.927744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927675 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qthjt\" (UniqueName: \"kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.927744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927696 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.927744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927725 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.927844 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927765 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.927844 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.927796 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:42.951754 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:42.951724 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:19:43.029156 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029127 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029167 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029327 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029294 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029401 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029337 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qthjt\" (UniqueName: \"kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029401 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029371 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029486 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029410 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.029626 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.029606 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.030105 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.030081 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.031866 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.031838 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.031866 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.031853 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.032034 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.031879 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.044152 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.044129 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qthjt\" (UniqueName: \"kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt\") pod \"feast-simple-feast-remote-setup-5485c76f7f-n9jbz\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.167634 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.167549 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:43.323195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:43.323161 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:19:43.327073 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:19:43.327039 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ebaa223_72bc_450e_b7b2_e678b6d7d839.slice/crio-613b720a7967256fbc9570e7092d8962e3480c2ead261323d45c54ca078e58af WatchSource:0}: Error finding container 613b720a7967256fbc9570e7092d8962e3480c2ead261323d45c54ca078e58af: Status 404 returned error can't find the container with id 613b720a7967256fbc9570e7092d8962e3480c2ead261323d45c54ca078e58af Jun 08 12:19:44.019768 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:44.019731 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"55875baf8ffdb7dc4b05a4765f6222fc3d5225c834718362bd180ca52258ecb3"} Jun 08 12:19:44.020139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:44.019780 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"613b720a7967256fbc9570e7092d8962e3480c2ead261323d45c54ca078e58af"} Jun 08 12:19:48.036501 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:48.036462 2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerID="55875baf8ffdb7dc4b05a4765f6222fc3d5225c834718362bd180ca52258ecb3" exitCode=0 Jun 08 12:19:48.036919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:48.036552 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"55875baf8ffdb7dc4b05a4765f6222fc3d5225c834718362bd180ca52258ecb3"} Jun 08 12:19:49.042159 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:49.042116 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"5a5725aadf30f569210b32f17148003ac0f068f4a76cb5937ee24e1ee9a980d3"} Jun 08 12:19:54.061205 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:54.061165 2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerID="5a5725aadf30f569210b32f17148003ac0f068f4a76cb5937ee24e1ee9a980d3" exitCode=0 Jun 08 12:19:54.061727 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:54.061244 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"5a5725aadf30f569210b32f17148003ac0f068f4a76cb5937ee24e1ee9a980d3"} Jun 08 12:19:55.068729 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.068694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"1c02bca68fa30d9db399f1677b5ebbb316d102ad7b89aa0a357fc25483164a95"} Jun 08 12:19:55.069261 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.068747 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"9b89da346c32601a916e6a5abab7c8bd7063ef439f834ee0b4976553f966c9fa"} Jun 08 12:19:55.069261 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.068763 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerStarted","Data":"60a971b69d8117973f4bfc0958ca0ba62213130e66151d4638ec147fa56b5619"} Jun 08 12:19:55.124204 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.124141 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podStartSLOduration=13.124120534 podStartE2EDuration="13.124120534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:19:55.122211554 +0000 UTC m=+763.232467777" watchObservedRunningTime="2026-06-08 12:19:55.124120534 +0000 UTC m=+763.234376757" Jun 08 12:19:55.168378 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.168332 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:55.168568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.168469 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:55.168568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.168528 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:19:55.170547 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.170489 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Jun 08 12:19:55.170547 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.170531 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Jun 08 12:19:55.170678 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:55.170505 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused" Jun 08 12:19:58.168432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:58.168388 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.20:8816: connect: connection refused" Jun 08 12:19:58.168908 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:58.168389 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Jun 08 12:19:58.168908 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:19:58.168391 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Jun 08 12:20:01.169800 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.169767 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.170207 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.169944 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.170402 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.170375 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.170553 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.170416 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.170657 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.170636 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.170726 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.170690 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:01.173926 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:01.173906 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:02.099460 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:02.099432 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:20:02.102447 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:20:02.102426 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:21:03.893374 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:03.893338 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:21:03.893905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:03.893661 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" containerID="cri-o://60a971b69d8117973f4bfc0958ca0ba62213130e66151d4638ec147fa56b5619" gracePeriod=30 Jun 08 12:21:03.893905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:03.893691 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" containerID="cri-o://1c02bca68fa30d9db399f1677b5ebbb316d102ad7b89aa0a357fc25483164a95" gracePeriod=30 Jun 08 12:21:03.893905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:03.893721 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" containerID="cri-o://9b89da346c32601a916e6a5abab7c8bd7063ef439f834ee0b4976553f966c9fa" gracePeriod=30 Jun 08 12:21:04.024673 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:04.024641 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:21:04.024940 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:04.024916 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="registry" containerID="cri-o://8e32bdb95ea9e92976440f3e0d790f13510568af9494247ea35d0f7ffee40bd0" gracePeriod=30 Jun 08 12:21:04.025041 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:04.024989 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="offline" containerID="cri-o://de6fd2902cc7903dbc5d5238ec8b7c9aa896aa42b3b466d0ea6767281100de4d" gracePeriod=30 Jun 08 12:21:04.025099 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:04.024997 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" containerID="cri-o://d0c3a2d9a8a174f06f033a467c9d88881daa0493ec6147e20fedbe8d952ad858" gracePeriod=30 Jun 08 12:21:04.025145 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:04.024989 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" containerID="cri-o://749b05ff03b2590c9451f350ee7e2047801c91cb188d8eb48497f0390547df6d" gracePeriod=30 Jun 08 12:21:05.332334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.332304 2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerID="1c02bca68fa30d9db399f1677b5ebbb316d102ad7b89aa0a357fc25483164a95" exitCode=0 Jun 08 12:21:05.332334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.332329 2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerID="60a971b69d8117973f4bfc0958ca0ba62213130e66151d4638ec147fa56b5619" exitCode=0 Jun 08 12:21:05.332783 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.332383 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"1c02bca68fa30d9db399f1677b5ebbb316d102ad7b89aa0a357fc25483164a95"} Jun 08 12:21:05.332783 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.332427 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"60a971b69d8117973f4bfc0958ca0ba62213130e66151d4638ec147fa56b5619"} Jun 08 12:21:05.335020 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.335000 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="749b05ff03b2590c9451f350ee7e2047801c91cb188d8eb48497f0390547df6d" exitCode=0 Jun 08 12:21:05.335020 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.335016 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="d0c3a2d9a8a174f06f033a467c9d88881daa0493ec6147e20fedbe8d952ad858" exitCode=0 Jun 08 12:21:05.335167 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.335040 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"749b05ff03b2590c9451f350ee7e2047801c91cb188d8eb48497f0390547df6d"} Jun 08 12:21:05.335167 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:05.335063 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"d0c3a2d9a8a174f06f033a467c9d88881daa0493ec6147e20fedbe8d952ad858"} Jun 08 12:21:09.405793 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:09.405738 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Jun 08 12:21:09.777048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:09.776952 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Jun 08 12:21:11.170592 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:11.170513 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Jun 08 12:21:12.100459 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:12.100417 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Jun 08 12:21:19.405403 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:19.405354 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Jun 08 12:21:19.777458 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:19.777362 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Jun 08 12:21:21.170680 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:21.170633 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Jun 08 12:21:22.100601 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:22.100559 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Jun 08 12:21:29.406106 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:29.406051 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.19:8443: connect: connection refused" Jun 08 12:21:29.406688 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:29.406196 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:21:29.777578 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:29.777469 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" probeResult="failure" output="Get \"https://10.132.0.19:6567/health\": dial tcp 10.132.0.19:6567: connect: connection refused" Jun 08 12:21:29.777724 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:29.777656 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:21:31.170910 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:31.170865 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.20:8443: connect: connection refused" Jun 08 12:21:31.171326 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:31.171005 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:21:32.100174 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:32.100131 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" probeResult="failure" output="Get \"https://10.132.0.20:6567/health\": dial tcp 10.132.0.20:6567: connect: connection refused" Jun 08 12:21:32.100354 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:32.100245 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:21:34.442351 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.442314 2576 generic.go:356] "Generic (PLEG): container finished" podID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerID="9b89da346c32601a916e6a5abab7c8bd7063ef439f834ee0b4976553f966c9fa" exitCode=137 Jun 08 12:21:34.442837 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.442391 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"9b89da346c32601a916e6a5abab7c8bd7063ef439f834ee0b4976553f966c9fa"} Jun 08 12:21:34.445964 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.445929 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="de6fd2902cc7903dbc5d5238ec8b7c9aa896aa42b3b466d0ea6767281100de4d" exitCode=137 Jun 08 12:21:34.445964 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.445955 2576 generic.go:356] "Generic (PLEG): container finished" podID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerID="8e32bdb95ea9e92976440f3e0d790f13510568af9494247ea35d0f7ffee40bd0" exitCode=137 Jun 08 12:21:34.446119 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.445993 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"de6fd2902cc7903dbc5d5238ec8b7c9aa896aa42b3b466d0ea6767281100de4d"} Jun 08 12:21:34.446119 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.446031 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"8e32bdb95ea9e92976440f3e0d790f13510568af9494247ea35d0f7ffee40bd0"} Jun 08 12:21:34.542351 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.542291 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:21:34.650985 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.650946 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651157 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651004 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qthjt\" (UniqueName: \"kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651157 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651028 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651157 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651076 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651217 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651298 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls\") pod \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\" (UID: \"2ebaa223-72bc-450e-b7b2-e678b6d7d839\") " Jun 08 12:21:34.651451 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651424 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data" (OuterVolumeSpecName: "feast-data") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:21:34.651726 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651702 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ebaa223-72bc-450e-b7b2-e678b6d7d839-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.651864 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.651718 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 12:21:34.655593 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.655570 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls" (OuterVolumeSpecName: "online-tls") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.655718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.655627 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.655718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.655680 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.655718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.655683 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt" (OuterVolumeSpecName: "kube-api-access-qthjt") pod "2ebaa223-72bc-450e-b7b2-e678b6d7d839" (UID: "2ebaa223-72bc-450e-b7b2-e678b6d7d839"). InnerVolumeSpecName "kube-api-access-qthjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:21:34.678816 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.678797 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:21:34.752539 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752491 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752550 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skwsv\" (UniqueName: \"kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752575 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752619 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752641 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752718 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752679 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls\") pod \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\" (UID: \"fb84736a-ca8f-4b43-aad0-c1b145deeab8\") " Jun 08 12:21:34.752989 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752844 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/2ebaa223-72bc-450e-b7b2-e678b6d7d839-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.752989 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752860 2576 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-offline-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.752989 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752874 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qthjt\" (UniqueName: \"kubernetes.io/projected/2ebaa223-72bc-450e-b7b2-e678b6d7d839-kube-api-access-qthjt\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.752989 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752887 2576 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-ui-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.752989 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.752901 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ebaa223-72bc-450e-b7b2-e678b6d7d839-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.753261 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.753195 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data" (OuterVolumeSpecName: "feast-data") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:21:34.754795 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.754758 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls" (OuterVolumeSpecName: "online-tls") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.754795 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.754772 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv" (OuterVolumeSpecName: "kube-api-access-skwsv") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "kube-api-access-skwsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:21:34.754795 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.754781 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.755161 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.755040 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.755161 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.755065 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fb84736a-ca8f-4b43-aad0-c1b145deeab8" (UID: "fb84736a-ca8f-4b43-aad0-c1b145deeab8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:21:34.853981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.853933 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skwsv\" (UniqueName: \"kubernetes.io/projected/fb84736a-ca8f-4b43-aad0-c1b145deeab8-kube-api-access-skwsv\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.853981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.853976 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/fb84736a-ca8f-4b43-aad0-c1b145deeab8-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.853981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.853986 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.853981 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.853998 2576 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-offline-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.854221 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.854006 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:34.854221 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:34.854014 2576 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/fb84736a-ca8f-4b43-aad0-c1b145deeab8-ui-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:21:35.452312 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.452281 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" Jun 08 12:21:35.452312 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.452299 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz" event={"ID":"fb84736a-ca8f-4b43-aad0-c1b145deeab8","Type":"ContainerDied","Data":"23d8ac193516742fedfbcf771c9ade0cbeac7b6c76718289c63d8460b084919d"} Jun 08 12:21:35.452883 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.452365 2576 scope.go:117] "RemoveContainer" containerID="749b05ff03b2590c9451f350ee7e2047801c91cb188d8eb48497f0390547df6d" Jun 08 12:21:35.455052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.455028 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" event={"ID":"2ebaa223-72bc-450e-b7b2-e678b6d7d839","Type":"ContainerDied","Data":"613b720a7967256fbc9570e7092d8962e3480c2ead261323d45c54ca078e58af"} Jun 08 12:21:35.455158 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.455087 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz" Jun 08 12:21:35.461357 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.461336 2576 scope.go:117] "RemoveContainer" containerID="de6fd2902cc7903dbc5d5238ec8b7c9aa896aa42b3b466d0ea6767281100de4d" Jun 08 12:21:35.473215 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.473194 2576 scope.go:117] "RemoveContainer" containerID="d0c3a2d9a8a174f06f033a467c9d88881daa0493ec6147e20fedbe8d952ad858" Jun 08 12:21:35.481410 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.481385 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:21:35.481784 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.481678 2576 scope.go:117] "RemoveContainer" containerID="8e32bdb95ea9e92976440f3e0d790f13510568af9494247ea35d0f7ffee40bd0" Jun 08 12:21:35.483757 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.483738 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-c55dcf4bd-lh2nz"] Jun 08 12:21:35.489679 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.489660 2576 scope.go:117] "RemoveContainer" containerID="b047be27e1c0ff4bd126382928f62ff2738a48bc284cf0f00dfa26dec1725b66" Jun 08 12:21:35.495305 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.495287 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:21:35.496887 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.496871 2576 scope.go:117] "RemoveContainer" containerID="7f617abc13cb3901fba4e927c68b34e4b76190064c4443bc11c6213508ccf39b" Jun 08 12:21:35.501070 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.501051 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5485c76f7f-n9jbz"] Jun 08 12:21:35.503889 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.503874 2576 scope.go:117] "RemoveContainer" containerID="1c02bca68fa30d9db399f1677b5ebbb316d102ad7b89aa0a357fc25483164a95" Jun 08 12:21:35.510706 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.510691 2576 scope.go:117] "RemoveContainer" containerID="9b89da346c32601a916e6a5abab7c8bd7063ef439f834ee0b4976553f966c9fa" Jun 08 12:21:35.518512 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.518497 2576 scope.go:117] "RemoveContainer" containerID="60a971b69d8117973f4bfc0958ca0ba62213130e66151d4638ec147fa56b5619" Jun 08 12:21:35.525681 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.525660 2576 scope.go:117] "RemoveContainer" containerID="5a5725aadf30f569210b32f17148003ac0f068f4a76cb5937ee24e1ee9a980d3" Jun 08 12:21:35.532136 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:35.532118 2576 scope.go:117] "RemoveContainer" containerID="55875baf8ffdb7dc4b05a4765f6222fc3d5225c834718362bd180ca52258ecb3" Jun 08 12:21:36.345680 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:36.345651 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" path="/var/lib/kubelet/pods/2ebaa223-72bc-450e-b7b2-e678b6d7d839/volumes" Jun 08 12:21:36.346185 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:36.346173 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" path="/var/lib/kubelet/pods/fb84736a-ca8f-4b43-aad0-c1b145deeab8/volumes" Jun 08 12:21:47.076705 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076675 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076942 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="feast-apply" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076951 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="feast-apply" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076961 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076966 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076973 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="feast-init" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076979 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="feast-init" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076985 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076991 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.076995 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077000 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077006 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="feast-init" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077013 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="feast-init" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077025 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="registry" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077031 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="registry" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077038 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077043 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077053 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077058 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077066 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="feast-apply" Jun 08 12:21:47.077068 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077071 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="feast-apply" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077080 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="offline" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077085 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="offline" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077124 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="ui" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077131 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="ui" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077136 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="registry" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077143 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="online" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077149 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="offline" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077153 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb84736a-ca8f-4b43-aad0-c1b145deeab8" containerName="offline" Jun 08 12:21:47.077670 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.077158 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ebaa223-72bc-450e-b7b2-e678b6d7d839" containerName="online" Jun 08 12:21:47.081753 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.081735 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.086863 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.086840 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jun 08 12:21:47.087589 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.087564 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jun 08 12:21:47.087712 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.087583 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-hbkc7\"" Jun 08 12:21:47.088052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.087978 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jun 08 12:21:47.091933 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.091912 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:21:47.135645 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.135616 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:21:47.140301 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.140200 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:21:47.142102 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.142073 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjq7t\" (UniqueName: \"kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.142209 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.142143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.155591 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.155565 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:21:47.242742 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.242706 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.242905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.242773 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8tvr\" (UniqueName: \"kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr\") pod \"redis-69bbd798b6-2ddhb\" (UID: \"f263df68-991e-4490-ab36-251cd237683a\") " pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:21:47.242905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.242802 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjq7t\" (UniqueName: \"kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.243173 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.243149 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.253300 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.253274 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjq7t\" (UniqueName: \"kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t\") pod \"postgres-864846d4f5-wc9lb\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.343979 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.343902 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8tvr\" (UniqueName: \"kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr\") pod \"redis-69bbd798b6-2ddhb\" (UID: \"f263df68-991e-4490-ab36-251cd237683a\") " pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:21:47.353893 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.353864 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8tvr\" (UniqueName: \"kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr\") pod \"redis-69bbd798b6-2ddhb\" (UID: \"f263df68-991e-4490-ab36-251cd237683a\") " pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:21:47.390681 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.390658 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:21:47.450881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.450844 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:21:47.523310 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.523278 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:21:47.526006 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:21:47.525976 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bb89144_56a6_4907_918e_c9d04c4d8c2a.slice/crio-3f16cbae158e8792f8cd8a0d6398ac7ec34b01ddd0e6d7d8a9e8edec36f8a042 WatchSource:0}: Error finding container 3f16cbae158e8792f8cd8a0d6398ac7ec34b01ddd0e6d7d8a9e8edec36f8a042: Status 404 returned error can't find the container with id 3f16cbae158e8792f8cd8a0d6398ac7ec34b01ddd0e6d7d8a9e8edec36f8a042 Jun 08 12:21:47.527799 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.527779 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:21:47.580311 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:47.580288 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:21:47.582667 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:21:47.582639 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf263df68_991e_4490_ab36_251cd237683a.slice/crio-069a9db0d46a4d276410a1b4b58a354c681a872917493dad54b1e5f661c54ca1 WatchSource:0}: Error finding container 069a9db0d46a4d276410a1b4b58a354c681a872917493dad54b1e5f661c54ca1: Status 404 returned error can't find the container with id 069a9db0d46a4d276410a1b4b58a354c681a872917493dad54b1e5f661c54ca1 Jun 08 12:21:48.505842 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:48.505803 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wc9lb" event={"ID":"6bb89144-56a6-4907-918e-c9d04c4d8c2a","Type":"ContainerStarted","Data":"3f16cbae158e8792f8cd8a0d6398ac7ec34b01ddd0e6d7d8a9e8edec36f8a042"} Jun 08 12:21:48.507268 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:48.507220 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-2ddhb" event={"ID":"f263df68-991e-4490-ab36-251cd237683a","Type":"ContainerStarted","Data":"069a9db0d46a4d276410a1b4b58a354c681a872917493dad54b1e5f661c54ca1"} Jun 08 12:21:49.512216 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:49.512181 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-2ddhb" event={"ID":"f263df68-991e-4490-ab36-251cd237683a","Type":"ContainerStarted","Data":"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c"} Jun 08 12:21:49.531371 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:49.531320 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-2ddhb" podStartSLOduration=0.725318753 podStartE2EDuration="2.531301826s" totalImagesPullingTime="1.805983073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:21:49.530634245 +0000 UTC m=+877.640890469" watchObservedRunningTime="2026-06-08 12:21:49.531301826 +0000 UTC m=+877.641558048" Jun 08 12:21:53.529209 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:53.529113 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wc9lb" event={"ID":"6bb89144-56a6-4907-918e-c9d04c4d8c2a","Type":"ContainerStarted","Data":"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663"} Jun 08 12:21:53.546019 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:53.545832 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-wc9lb" podStartSLOduration=0.905992625 podStartE2EDuration="6.545813445s" totalImagesPullingTime="5.63982082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:21:53.54533338 +0000 UTC m=+881.655589595" watchObservedRunningTime="2026-06-08 12:21:53.545813445 +0000 UTC m=+881.656069668" Jun 08 12:21:57.755480 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.755400 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:21:57.761023 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.760994 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:57.763788 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.763764 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jun 08 12:21:57.763788 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.763783 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-vtfsk\"" Jun 08 12:21:57.763954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.763770 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jun 08 12:21:57.763954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.763787 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jun 08 12:21:57.763954 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.763917 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jun 08 12:21:57.771124 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.771103 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:21:57.928393 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.928364 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:57.928612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.928400 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:57.928612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.928419 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:57.928612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.928440 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5s54\" (UniqueName: \"kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:57.928612 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:57.928569 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029240 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029161 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029240 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029196 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029240 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029219 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5s54\" (UniqueName: \"kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029493 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029254 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029493 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029293 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.029760 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.029733 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.031755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.031733 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.031856 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.031780 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.031856 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.031801 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.037252 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.037233 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5s54\" (UniqueName: \"kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54\") pod \"feast-credit-scoring-55c4cc5dfd-wjqlm\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.069927 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.069896 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:21:58.204277 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.204243 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:21:58.207371 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:21:58.207347 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800121e4_8a0a_4ef4_8259_a3f46a910a52.slice/crio-c022e36c8360888797c5710c3b08d913d6735c01abdab9d2175dc74b382da91d WatchSource:0}: Error finding container c022e36c8360888797c5710c3b08d913d6735c01abdab9d2175dc74b382da91d: Status 404 returned error can't find the container with id c022e36c8360888797c5710c3b08d913d6735c01abdab9d2175dc74b382da91d Jun 08 12:21:58.548476 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.548439 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac"} Jun 08 12:21:58.548665 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:58.548624 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"c022e36c8360888797c5710c3b08d913d6735c01abdab9d2175dc74b382da91d"} Jun 08 12:21:59.553133 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:59.553099 2576 generic.go:356] "Generic (PLEG): container finished" podID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerID="7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac" exitCode=0 Jun 08 12:21:59.553620 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:21:59.553170 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac"} Jun 08 12:22:00.557995 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:00.557956 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368"} Jun 08 12:22:06.583750 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:06.583713 2576 generic.go:356] "Generic (PLEG): container finished" podID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerID="618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368" exitCode=0 Jun 08 12:22:06.584140 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:06.583768 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368"} Jun 08 12:22:07.593091 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:07.593049 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579"} Jun 08 12:22:07.593472 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:07.593125 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef"} Jun 08 12:22:07.593472 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:07.593152 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerStarted","Data":"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e"} Jun 08 12:22:07.624473 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:07.624407 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podStartSLOduration=10.624381044 podStartE2EDuration="10.624381044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:21:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:22:07.622631758 +0000 UTC m=+895.732888022" watchObservedRunningTime="2026-06-08 12:22:07.624381044 +0000 UTC m=+895.734637267" Jun 08 12:22:08.070220 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:08.070180 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:08.070393 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:08.070258 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:08.070393 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:08.070287 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:10.071093 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.071051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:10.071563 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.071277 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:10.071563 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.071312 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:10.072840 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.072801 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.23:8816: connect: connection refused" Jun 08 12:22:10.072840 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.072822 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused" Jun 08 12:22:10.073020 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:10.072865 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jun 08 12:22:13.074633 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:13.074566 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused" Jun 08 12:22:13.075466 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:13.075436 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:13.077778 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:13.077754 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:13.080394 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:13.080368 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:13.623062 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:13.623023 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:16.070916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:16.070870 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.23:6571: connect: connection refused" Jun 08 12:22:19.071478 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:19.071442 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:22:19.072770 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:22:19.072155 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:23:21.932276 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.932237 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:23:21.932801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.932540 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" containerID="cri-o://c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e" gracePeriod=30 Jun 08 12:23:21.932801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.932583 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" containerID="cri-o://4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef" gracePeriod=30 Jun 08 12:23:21.932801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.932582 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="offline" containerID="cri-o://01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579" gracePeriod=30 Jun 08 12:23:21.963303 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.963268 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:23:21.963509 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.963483 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-wc9lb" podUID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" containerName="postgres" containerID="cri-o://329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663" gracePeriod=30 Jun 08 12:23:21.974785 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.974747 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:23:21.975047 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:21.975000 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-2ddhb" podUID="f263df68-991e-4490-ab36-251cd237683a" containerName="redis" containerID="cri-o://c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c" gracePeriod=30 Jun 08 12:23:22.237390 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.237367 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:23:22.240341 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.240322 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:23:22.398405 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.398366 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8tvr\" (UniqueName: \"kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr\") pod \"f263df68-991e-4490-ab36-251cd237683a\" (UID: \"f263df68-991e-4490-ab36-251cd237683a\") " Jun 08 12:23:22.398643 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.398418 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata\") pod \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " Jun 08 12:23:22.398643 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.398442 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjq7t\" (UniqueName: \"kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t\") pod \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\" (UID: \"6bb89144-56a6-4907-918e-c9d04c4d8c2a\") " Jun 08 12:23:22.400587 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.400549 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr" (OuterVolumeSpecName: "kube-api-access-z8tvr") pod "f263df68-991e-4490-ab36-251cd237683a" (UID: "f263df68-991e-4490-ab36-251cd237683a"). InnerVolumeSpecName "kube-api-access-z8tvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:23:22.400587 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.400564 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t" (OuterVolumeSpecName: "kube-api-access-gjq7t") pod "6bb89144-56a6-4907-918e-c9d04c4d8c2a" (UID: "6bb89144-56a6-4907-918e-c9d04c4d8c2a"). InnerVolumeSpecName "kube-api-access-gjq7t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:23:22.427644 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.427595 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "6bb89144-56a6-4907-918e-c9d04c4d8c2a" (UID: "6bb89144-56a6-4907-918e-c9d04c4d8c2a"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:23:22.499762 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.499678 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8tvr\" (UniqueName: \"kubernetes.io/projected/f263df68-991e-4490-ab36-251cd237683a-kube-api-access-z8tvr\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:22.499762 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.499709 2576 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6bb89144-56a6-4907-918e-c9d04c4d8c2a-postgresdata\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:22.499762 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.499720 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjq7t\" (UniqueName: \"kubernetes.io/projected/6bb89144-56a6-4907-918e-c9d04c4d8c2a-kube-api-access-gjq7t\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:22.887426 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.887393 2576 generic.go:356] "Generic (PLEG): container finished" podID="f263df68-991e-4490-ab36-251cd237683a" containerID="c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c" exitCode=0 Jun 08 12:23:22.887664 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.887439 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-2ddhb" Jun 08 12:23:22.887664 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.887474 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-2ddhb" event={"ID":"f263df68-991e-4490-ab36-251cd237683a","Type":"ContainerDied","Data":"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c"} Jun 08 12:23:22.887664 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.887508 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-2ddhb" event={"ID":"f263df68-991e-4490-ab36-251cd237683a","Type":"ContainerDied","Data":"069a9db0d46a4d276410a1b4b58a354c681a872917493dad54b1e5f661c54ca1"} Jun 08 12:23:22.887664 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.887547 2576 scope.go:117] "RemoveContainer" containerID="c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c" Jun 08 12:23:22.890380 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.890357 2576 generic.go:356] "Generic (PLEG): container finished" podID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerID="4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef" exitCode=0 Jun 08 12:23:22.890485 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.890419 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef"} Jun 08 12:23:22.891502 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.891479 2576 generic.go:356] "Generic (PLEG): container finished" podID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" containerID="329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663" exitCode=0 Jun 08 12:23:22.891622 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.891509 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wc9lb" Jun 08 12:23:22.891622 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.891546 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wc9lb" event={"ID":"6bb89144-56a6-4907-918e-c9d04c4d8c2a","Type":"ContainerDied","Data":"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663"} Jun 08 12:23:22.891622 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.891573 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wc9lb" event={"ID":"6bb89144-56a6-4907-918e-c9d04c4d8c2a","Type":"ContainerDied","Data":"3f16cbae158e8792f8cd8a0d6398ac7ec34b01ddd0e6d7d8a9e8edec36f8a042"} Jun 08 12:23:22.896490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.896468 2576 scope.go:117] "RemoveContainer" containerID="c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c" Jun 08 12:23:22.896743 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:22.896720 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c\": container with ID starting with c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c not found: ID does not exist" containerID="c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c" Jun 08 12:23:22.896839 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.896750 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c"} err="failed to get container status \"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c\": rpc error: code = NotFound desc = could not find container \"c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c\": container with ID starting with c89080bbeef1b6280e244ec8e61736ac758088dddad3c9fe11d3aae3f3ad613c not found: ID does not exist" Jun 08 12:23:22.896839 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.896771 2576 scope.go:117] "RemoveContainer" containerID="329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663" Jun 08 12:23:22.904465 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.904448 2576 scope.go:117] "RemoveContainer" containerID="329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663" Jun 08 12:23:22.904737 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:22.904718 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663\": container with ID starting with 329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663 not found: ID does not exist" containerID="329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663" Jun 08 12:23:22.904782 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.904744 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663"} err="failed to get container status \"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663\": rpc error: code = NotFound desc = could not find container \"329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663\": container with ID starting with 329f0b5055683541020120d49a255d6c13a3dee5be783fcbb6cfb1019922e663 not found: ID does not exist" Jun 08 12:23:22.913434 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.913409 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:23:22.920352 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.920333 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-2ddhb"] Jun 08 12:23:22.935904 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.935875 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:23:22.939449 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:22.939429 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wc9lb"] Jun 08 12:23:23.620219 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:23.620174 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jun 08 12:23:24.345061 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:24.345014 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" path="/var/lib/kubelet/pods/6bb89144-56a6-4907-918e-c9d04c4d8c2a/volumes" Jun 08 12:23:24.345448 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:24.345353 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f263df68-991e-4490-ab36-251cd237683a" path="/var/lib/kubelet/pods/f263df68-991e-4490-ab36-251cd237683a/volumes" Jun 08 12:23:33.620027 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:33.619929 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jun 08 12:23:43.620970 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:43.620924 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" probeResult="failure" output="Get \"https://10.132.0.23:6567/health\": dial tcp 10.132.0.23:6567: connect: connection refused" Jun 08 12:23:43.621400 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:43.621062 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:23:52.003581 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.003540 2576 generic.go:356] "Generic (PLEG): container finished" podID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerID="01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579" exitCode=137 Jun 08 12:23:52.003900 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.003547 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579"} Jun 08 12:23:52.596978 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.596952 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:23:52.717895 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.717855 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5s54\" (UniqueName: \"kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54\") pod \"800121e4-8a0a-4ef4-8259-a3f46a910a52\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " Jun 08 12:23:52.718123 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.717934 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls\") pod \"800121e4-8a0a-4ef4-8259-a3f46a910a52\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " Jun 08 12:23:52.718123 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.717953 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls\") pod \"800121e4-8a0a-4ef4-8259-a3f46a910a52\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " Jun 08 12:23:52.718123 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.717976 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data\") pod \"800121e4-8a0a-4ef4-8259-a3f46a910a52\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " Jun 08 12:23:52.718123 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.718015 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls\") pod \"800121e4-8a0a-4ef4-8259-a3f46a910a52\" (UID: \"800121e4-8a0a-4ef4-8259-a3f46a910a52\") " Jun 08 12:23:52.720172 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.720144 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54" (OuterVolumeSpecName: "kube-api-access-g5s54") pod "800121e4-8a0a-4ef4-8259-a3f46a910a52" (UID: "800121e4-8a0a-4ef4-8259-a3f46a910a52"). InnerVolumeSpecName "kube-api-access-g5s54". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:23:52.720172 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.720158 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "800121e4-8a0a-4ef4-8259-a3f46a910a52" (UID: "800121e4-8a0a-4ef4-8259-a3f46a910a52"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:23:52.720320 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.720184 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "800121e4-8a0a-4ef4-8259-a3f46a910a52" (UID: "800121e4-8a0a-4ef4-8259-a3f46a910a52"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:23:52.720359 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.720323 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls" (OuterVolumeSpecName: "online-tls") pod "800121e4-8a0a-4ef4-8259-a3f46a910a52" (UID: "800121e4-8a0a-4ef4-8259-a3f46a910a52"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:23:52.726392 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.726366 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data" (OuterVolumeSpecName: "feast-data") pod "800121e4-8a0a-4ef4-8259-a3f46a910a52" (UID: "800121e4-8a0a-4ef4-8259-a3f46a910a52"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:23:52.819185 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.819147 2576 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-offline-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:52.819185 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.819177 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5s54\" (UniqueName: \"kubernetes.io/projected/800121e4-8a0a-4ef4-8259-a3f46a910a52-kube-api-access-g5s54\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:52.819185 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.819188 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:52.819185 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.819196 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/800121e4-8a0a-4ef4-8259-a3f46a910a52-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:52.819455 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:52.819204 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/800121e4-8a0a-4ef4-8259-a3f46a910a52-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:23:53.009706 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.009620 2576 generic.go:356] "Generic (PLEG): container finished" podID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerID="c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e" exitCode=137 Jun 08 12:23:53.010134 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.009888 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e"} Jun 08 12:23:53.010134 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.009912 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" Jun 08 12:23:53.010134 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.009967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm" event={"ID":"800121e4-8a0a-4ef4-8259-a3f46a910a52","Type":"ContainerDied","Data":"c022e36c8360888797c5710c3b08d913d6735c01abdab9d2175dc74b382da91d"} Jun 08 12:23:53.010134 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.010004 2576 scope.go:117] "RemoveContainer" containerID="01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579" Jun 08 12:23:53.019332 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.019314 2576 scope.go:117] "RemoveContainer" containerID="4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef" Jun 08 12:23:53.026491 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.026474 2576 scope.go:117] "RemoveContainer" containerID="c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e" Jun 08 12:23:53.033799 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.033778 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:23:53.033994 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.033977 2576 scope.go:117] "RemoveContainer" containerID="618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368" Jun 08 12:23:53.040039 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.040015 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-55c4cc5dfd-wjqlm"] Jun 08 12:23:53.041680 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.041664 2576 scope.go:117] "RemoveContainer" containerID="7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac" Jun 08 12:23:53.048194 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048176 2576 scope.go:117] "RemoveContainer" containerID="01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579" Jun 08 12:23:53.048433 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:53.048414 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579\": container with ID starting with 01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579 not found: ID does not exist" containerID="01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579" Jun 08 12:23:53.048490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048441 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579"} err="failed to get container status \"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579\": rpc error: code = NotFound desc = could not find container \"01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579\": container with ID starting with 01803ce2def6ffbc81022048b19543a78a2e5784fd8da7440a72271863622579 not found: ID does not exist" Jun 08 12:23:53.048490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048460 2576 scope.go:117] "RemoveContainer" containerID="4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef" Jun 08 12:23:53.048710 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:53.048695 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef\": container with ID starting with 4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef not found: ID does not exist" containerID="4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef" Jun 08 12:23:53.048750 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048714 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef"} err="failed to get container status \"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef\": rpc error: code = NotFound desc = could not find container \"4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef\": container with ID starting with 4c19d4b09ea54e058821ef2a46e2cfc19d44921caadef14d42f6c1c6752f0cef not found: ID does not exist" Jun 08 12:23:53.048750 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048727 2576 scope.go:117] "RemoveContainer" containerID="c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e" Jun 08 12:23:53.048946 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:53.048930 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e\": container with ID starting with c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e not found: ID does not exist" containerID="c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e" Jun 08 12:23:53.048990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048950 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e"} err="failed to get container status \"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e\": rpc error: code = NotFound desc = could not find container \"c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e\": container with ID starting with c12715e9ae3628dd54a5d1fad62f7955816a15bb9a8168340094cdd6114f621e not found: ID does not exist" Jun 08 12:23:53.048990 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.048963 2576 scope.go:117] "RemoveContainer" containerID="618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368" Jun 08 12:23:53.049178 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:53.049161 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368\": container with ID starting with 618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368 not found: ID does not exist" containerID="618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368" Jun 08 12:23:53.049215 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.049185 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368"} err="failed to get container status \"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368\": rpc error: code = NotFound desc = could not find container \"618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368\": container with ID starting with 618c3cd6b1068f34523affb2d7032993c749e69b1bff81c91da47d3c7ae24368 not found: ID does not exist" Jun 08 12:23:53.049215 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.049200 2576 scope.go:117] "RemoveContainer" containerID="7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac" Jun 08 12:23:53.049428 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:23:53.049412 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac\": container with ID starting with 7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac not found: ID does not exist" containerID="7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac" Jun 08 12:23:53.049469 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:53.049431 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac"} err="failed to get container status \"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac\": rpc error: code = NotFound desc = could not find container \"7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac\": container with ID starting with 7a8afcc6fd46979738f7a7c8b69709769af374fbc81c2e7d0d9b8d1a2b8a37ac not found: ID does not exist" Jun 08 12:23:54.345606 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:23:54.345574 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" path="/var/lib/kubelet/pods/800121e4-8a0a-4ef4-8259-a3f46a910a52/volumes" Jun 08 12:25:35.198198 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198120 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198371 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="offline" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198382 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="offline" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198391 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="feast-init" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198397 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="feast-init" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198408 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" containerName="postgres" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198414 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" containerName="postgres" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198423 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198428 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198440 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f263df68-991e-4490-ab36-251cd237683a" containerName="redis" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198444 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f263df68-991e-4490-ab36-251cd237683a" containerName="redis" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198450 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="feast-apply" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198454 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="feast-apply" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198460 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198465 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198503 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="online" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198510 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f263df68-991e-4490-ab36-251cd237683a" containerName="redis" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198532 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="offline" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198542 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="800121e4-8a0a-4ef4-8259-a3f46a910a52" containerName="registry" Jun 08 12:25:35.198677 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.198548 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bb89144-56a6-4907-918e-c9d04c4d8c2a" containerName="postgres" Jun 08 12:25:35.201245 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.201223 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:25:35.203875 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.203840 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jun 08 12:25:35.205028 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.205011 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-fm5wt\"" Jun 08 12:25:35.205128 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.205012 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jun 08 12:25:35.217565 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.217534 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:25:35.243567 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.243534 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-475wz\" (UniqueName: \"kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz\") pod \"redis-69bbd798b6-tqvwp\" (UID: \"3399c92c-e233-4003-88cc-585c696dd23f\") " pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:25:35.343801 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.343772 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-475wz\" (UniqueName: \"kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz\") pod \"redis-69bbd798b6-tqvwp\" (UID: \"3399c92c-e233-4003-88cc-585c696dd23f\") " pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:25:35.353407 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.353385 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-475wz\" (UniqueName: \"kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz\") pod \"redis-69bbd798b6-tqvwp\" (UID: \"3399c92c-e233-4003-88cc-585c696dd23f\") " pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:25:35.510441 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.510361 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:25:35.647344 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:35.647312 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:25:35.650542 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:25:35.650496 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3399c92c_e233_4003_88cc_585c696dd23f.slice/crio-b16171a6cdad7e8b21653596d1ae1ee7b843baaba202bfa119b3bcaff3313943 WatchSource:0}: Error finding container b16171a6cdad7e8b21653596d1ae1ee7b843baaba202bfa119b3bcaff3313943: Status 404 returned error can't find the container with id b16171a6cdad7e8b21653596d1ae1ee7b843baaba202bfa119b3bcaff3313943 Jun 08 12:25:36.364113 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:36.364080 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" event={"ID":"3399c92c-e233-4003-88cc-585c696dd23f","Type":"ContainerStarted","Data":"0e64ad73e4c8583d55755aadaf20a23357730788934bfe760a0566ec0a66ef90"} Jun 08 12:25:36.364507 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:36.364127 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" event={"ID":"3399c92c-e233-4003-88cc-585c696dd23f","Type":"ContainerStarted","Data":"b16171a6cdad7e8b21653596d1ae1ee7b843baaba202bfa119b3bcaff3313943"} Jun 08 12:25:36.380257 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:36.380207 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" podStartSLOduration=1.380189542 podStartE2EDuration="1.380189542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:25:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:25:36.379867973 +0000 UTC m=+1104.490124196" watchObservedRunningTime="2026-06-08 12:25:36.380189542 +0000 UTC m=+1104.490445759" Jun 08 12:25:40.685851 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.685819 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:25:40.688720 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.688698 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.692034 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.692010 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jun 08 12:25:40.700782 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.700758 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:25:40.782686 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.782659 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.782825 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.782695 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjvhk\" (UniqueName: \"kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.883627 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.883600 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.883744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.883636 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjvhk\" (UniqueName: \"kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.884025 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.884007 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.893069 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.893047 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjvhk\" (UniqueName: \"kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk\") pod \"postgres-864846d4f5-dsflg\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:40.998611 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:40.998504 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:25:41.123095 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:41.123069 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:25:41.128112 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:25:41.128052 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3103f6e0_9acb_4d24_aa82_ac6a0c8f6ac9.slice/crio-ac4cd414aeefbb3f26ad4aae6aaad06692747d53eaa20f8886bc6a841cf17ec6 WatchSource:0}: Error finding container ac4cd414aeefbb3f26ad4aae6aaad06692747d53eaa20f8886bc6a841cf17ec6: Status 404 returned error can't find the container with id ac4cd414aeefbb3f26ad4aae6aaad06692747d53eaa20f8886bc6a841cf17ec6 Jun 08 12:25:41.384488 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:41.384453 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" event={"ID":"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9","Type":"ContainerStarted","Data":"57b417a1d633ebe7081c312db4ddfbd3c225b008283d84e796dcca41aa649212"} Jun 08 12:25:41.384660 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:41.384498 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" event={"ID":"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9","Type":"ContainerStarted","Data":"ac4cd414aeefbb3f26ad4aae6aaad06692747d53eaa20f8886bc6a841cf17ec6"} Jun 08 12:25:41.405142 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:41.405082 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" podStartSLOduration=1.405067615 podStartE2EDuration="1.405067615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:25:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:25:41.404155645 +0000 UTC m=+1109.514411889" watchObservedRunningTime="2026-06-08 12:25:41.405067615 +0000 UTC m=+1109.515323837" Jun 08 12:25:46.390120 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.390084 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:25:46.393595 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.393573 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.397212 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.397190 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jun 08 12:25:46.397338 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.397286 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jun 08 12:25:46.397674 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.397653 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jun 08 12:25:46.397789 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.397723 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-hmmfr\"" Jun 08 12:25:46.417183 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.417155 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:25:46.424722 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.424691 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.424832 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.424750 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.424832 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.424770 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl9gt\" (UniqueName: \"kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.424952 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.424926 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.525873 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.525832 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.526052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.525886 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.526052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.525905 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl9gt\" (UniqueName: \"kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.526052 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.525930 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.526334 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.526309 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.528291 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.528265 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.528386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.528305 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.535376 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.535355 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl9gt\" (UniqueName: \"kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt\") pod \"feast-credit-scoring-66b4d7c4d9-fxwlq\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.703010 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.702918 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:46.843739 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:46.843703 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:25:46.846871 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:25:46.846846 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3313bd2_599f_4767_a022_1a9cb52f9055.slice/crio-ef3d7a4e62d83fc320336eeb48d594d7567137597f1b5ad895e44d9adcb683ef WatchSource:0}: Error finding container ef3d7a4e62d83fc320336eeb48d594d7567137597f1b5ad895e44d9adcb683ef: Status 404 returned error can't find the container with id ef3d7a4e62d83fc320336eeb48d594d7567137597f1b5ad895e44d9adcb683ef Jun 08 12:25:47.406916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:47.406883 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerStarted","Data":"d64d62b911eb8e3257e8a7d9b0726b600bc51b61a43ca5ed45fbe01e9fa52cd2"} Jun 08 12:25:47.407314 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:47.406934 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerStarted","Data":"ef3d7a4e62d83fc320336eeb48d594d7567137597f1b5ad895e44d9adcb683ef"} Jun 08 12:25:48.411089 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:48.411053 2576 generic.go:356] "Generic (PLEG): container finished" podID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerID="d64d62b911eb8e3257e8a7d9b0726b600bc51b61a43ca5ed45fbe01e9fa52cd2" exitCode=0 Jun 08 12:25:48.411462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:48.411096 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerDied","Data":"d64d62b911eb8e3257e8a7d9b0726b600bc51b61a43ca5ed45fbe01e9fa52cd2"} Jun 08 12:25:49.419018 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:49.418978 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerStarted","Data":"5c15bbb5177cc11d95afe6bd8152472f0747d0e6ce39c23e490904b90eee14ce"} Jun 08 12:25:55.443841 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:55.443804 2576 generic.go:356] "Generic (PLEG): container finished" podID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerID="5c15bbb5177cc11d95afe6bd8152472f0747d0e6ce39c23e490904b90eee14ce" exitCode=0 Jun 08 12:25:55.443841 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:55.443847 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerDied","Data":"5c15bbb5177cc11d95afe6bd8152472f0747d0e6ce39c23e490904b90eee14ce"} Jun 08 12:25:56.450373 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:56.450341 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerStarted","Data":"0bd6e35548a1c114145268784cca543e45922d3ac29ecbf985bb9a21a570c498"} Jun 08 12:25:56.450848 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:56.450393 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerStarted","Data":"68ce74e02f8d4bb74fb7161d89fb5988b667fbd4965a75b496b53231a1b15df4"} Jun 08 12:25:56.492785 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:56.492723 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podStartSLOduration=10.492704941 podStartE2EDuration="10.492704941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:25:56.492396549 +0000 UTC m=+1124.602652791" watchObservedRunningTime="2026-06-08 12:25:56.492704941 +0000 UTC m=+1124.602961162" Jun 08 12:25:56.703568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:56.703451 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:56.703568 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:56.703529 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:58.703353 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:58.703309 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:58.703834 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:58.703388 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:25:58.705152 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:58.705121 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jun 08 12:25:58.705261 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:25:58.705147 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jun 08 12:26:01.703705 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:01.703643 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jun 08 12:26:01.708787 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:01.708760 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:26:01.711773 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:01.711748 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:26:04.703878 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:04.703817 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.26:6571: connect: connection refused" Jun 08 12:26:07.704130 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:07.704096 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:26:07.704838 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:07.704817 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:26:12.111743 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.111709 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:26:12.115148 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.115126 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.118142 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.118119 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-28d9r\"" Jun 08 12:26:12.118289 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.118119 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jun 08 12:26:12.118393 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.118364 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jun 08 12:26:12.118393 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.118371 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jun 08 12:26:12.129345 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.129314 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:26:12.231367 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.231326 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.231367 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.231376 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmsjw\" (UniqueName: \"kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.231626 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.231414 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.231626 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.231432 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.332231 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.332193 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.332404 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.332253 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmsjw\" (UniqueName: \"kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.332404 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.332279 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.332404 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.332295 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.332709 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.332690 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.335748 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.335727 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jun 08 12:26:12.335862 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.335842 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jun 08 12:26:12.342503 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.342473 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmsjw\" (UniqueName: \"kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.344877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.344852 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.345009 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.344892 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls\") pod \"feast-driver-ranking-6dbc7c5698-m9ks2\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.427866 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.427790 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-28d9r\"" Jun 08 12:26:12.435139 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.435109 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:12.580287 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:12.580252 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:26:12.584107 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:26:12.584075 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d5a183_e00d_4054_9d92_1473e9d4c42f.slice/crio-96dd8b4a63ee99ecf4e8c1dcebc524107cecdb865fc3fb343fe3a3532d24d241 WatchSource:0}: Error finding container 96dd8b4a63ee99ecf4e8c1dcebc524107cecdb865fc3fb343fe3a3532d24d241: Status 404 returned error can't find the container with id 96dd8b4a63ee99ecf4e8c1dcebc524107cecdb865fc3fb343fe3a3532d24d241 Jun 08 12:26:13.519000 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:13.518967 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerStarted","Data":"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6"} Jun 08 12:26:13.519383 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:13.519020 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerStarted","Data":"96dd8b4a63ee99ecf4e8c1dcebc524107cecdb865fc3fb343fe3a3532d24d241"} Jun 08 12:26:17.537080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:17.536987 2576 generic.go:356] "Generic (PLEG): container finished" podID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerID="7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6" exitCode=0 Jun 08 12:26:17.537080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:17.537041 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerDied","Data":"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6"} Jun 08 12:26:18.542450 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:18.542415 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerStarted","Data":"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b"} Jun 08 12:26:25.572370 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:25.572336 2576 generic.go:356] "Generic (PLEG): container finished" podID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerID="c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b" exitCode=0 Jun 08 12:26:25.572784 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:25.572417 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerDied","Data":"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b"} Jun 08 12:26:25.579911 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:25.579884 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jun 08 12:26:26.579731 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:26.579694 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerStarted","Data":"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4"} Jun 08 12:26:26.580117 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:26.579749 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerStarted","Data":"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877"} Jun 08 12:26:26.604275 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:26.604215 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podStartSLOduration=14.604195292 podStartE2EDuration="14.604195292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:26:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:26:26.602776424 +0000 UTC m=+1154.713032646" watchObservedRunningTime="2026-06-08 12:26:26.604195292 +0000 UTC m=+1154.714451577" Jun 08 12:26:27.436144 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:27.436092 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:27.436355 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:27.436186 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:27.437719 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:27.437673 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jun 08 12:26:27.437719 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:27.437700 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jun 08 12:26:30.436222 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:30.436169 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jun 08 12:26:30.436765 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:30.436259 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jun 08 12:26:32.435919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:32.435870 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:32.436315 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:32.435961 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:33.435763 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:33.435708 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jun 08 12:26:33.441195 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:33.441161 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:33.444397 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:33.444370 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:36.435955 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:36.435912 2576 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.27:6571: connect: connection refused" Jun 08 12:26:39.436780 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:39.436745 2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:26:39.437449 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:26:39.437430 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:27:56.450100 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.450061 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:27:56.452646 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.450259 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" podUID="8290a8ca-37af-4eca-9006-3a50fbc17a06" containerName="manager" containerID="cri-o://24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773" gracePeriod=10 Jun 08 12:27:56.707641 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.707573 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:27:56.771093 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.771053 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt9m9\" (UniqueName: \"kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9\") pod \"8290a8ca-37af-4eca-9006-3a50fbc17a06\" (UID: \"8290a8ca-37af-4eca-9006-3a50fbc17a06\") " Jun 08 12:27:56.773325 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.773287 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9" (OuterVolumeSpecName: "kube-api-access-zt9m9") pod "8290a8ca-37af-4eca-9006-3a50fbc17a06" (UID: "8290a8ca-37af-4eca-9006-3a50fbc17a06"). InnerVolumeSpecName "kube-api-access-zt9m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:27:56.872338 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.872307 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt9m9\" (UniqueName: \"kubernetes.io/projected/8290a8ca-37af-4eca-9006-3a50fbc17a06-kube-api-access-zt9m9\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:27:56.926531 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.926480 2576 generic.go:356] "Generic (PLEG): container finished" podID="8290a8ca-37af-4eca-9006-3a50fbc17a06" containerID="24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773" exitCode=0 Jun 08 12:27:56.926697 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.926540 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" Jun 08 12:27:56.926697 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.926576 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" event={"ID":"8290a8ca-37af-4eca-9006-3a50fbc17a06","Type":"ContainerDied","Data":"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773"} Jun 08 12:27:56.926697 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.926630 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb" event={"ID":"8290a8ca-37af-4eca-9006-3a50fbc17a06","Type":"ContainerDied","Data":"3065aea0409e5a5dd469b623ad79954fc02b94eedc78d729bca0e134080f7759"} Jun 08 12:27:56.926697 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.926660 2576 scope.go:117] "RemoveContainer" containerID="24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773" Jun 08 12:27:56.934896 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.934877 2576 scope.go:117] "RemoveContainer" containerID="24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773" Jun 08 12:27:56.935135 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:27:56.935118 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773\": container with ID starting with 24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773 not found: ID does not exist" containerID="24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773" Jun 08 12:27:56.935204 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.935148 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773"} err="failed to get container status \"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773\": rpc error: code = NotFound desc = could not find container \"24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773\": container with ID starting with 24303c09df6571c3bb543fc7b047512c9ab14069b05bca3ee5b68394e0c8d773 not found: ID does not exist" Jun 08 12:27:56.984490 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.984429 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:27:56.997610 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.997588 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:27:56.997837 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.997798 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" containerID="cri-o://68ce74e02f8d4bb74fb7161d89fb5988b667fbd4965a75b496b53231a1b15df4" gracePeriod=30 Jun 08 12:27:56.997929 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:56.997818 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" containerID="cri-o://0bd6e35548a1c114145268784cca543e45922d3ac29ecbf985bb9a21a570c498" gracePeriod=30 Jun 08 12:27:57.000200 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.000176 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-78576d7dd5-482xb"] Jun 08 12:27:57.019368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.019343 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:27:57.019657 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.019627 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" containerID="cri-o://13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877" gracePeriod=30 Jun 08 12:27:57.019790 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.019636 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" containerID="cri-o://8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4" gracePeriod=30 Jun 08 12:27:57.933384 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.933345 2576 generic.go:356] "Generic (PLEG): container finished" podID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerID="8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4" exitCode=0 Jun 08 12:27:57.933849 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.933420 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerDied","Data":"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4"} Jun 08 12:27:57.935532 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.935494 2576 generic.go:356] "Generic (PLEG): container finished" podID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerID="0bd6e35548a1c114145268784cca543e45922d3ac29ecbf985bb9a21a570c498" exitCode=0 Jun 08 12:27:57.935616 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:57.935568 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerDied","Data":"0bd6e35548a1c114145268784cca543e45922d3ac29ecbf985bb9a21a570c498"} Jun 08 12:27:58.346198 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.346166 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8290a8ca-37af-4eca-9006-3a50fbc17a06" path="/var/lib/kubelet/pods/8290a8ca-37af-4eca-9006-3a50fbc17a06/volumes" Jun 08 12:27:58.730244 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.730145 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:27:58.730408 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.730382 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" podUID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" containerName="postgres" containerID="cri-o://57b417a1d633ebe7081c312db4ddfbd3c225b008283d84e796dcca41aa649212" gracePeriod=30 Jun 08 12:27:58.740926 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.740899 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:27:58.741117 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.741082 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" podUID="3399c92c-e233-4003-88cc-585c696dd23f" containerName="redis" containerID="cri-o://0e64ad73e4c8583d55755aadaf20a23357730788934bfe760a0566ec0a66ef90" gracePeriod=30 Jun 08 12:27:58.943126 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.943096 2576 generic.go:356] "Generic (PLEG): container finished" podID="3399c92c-e233-4003-88cc-585c696dd23f" containerID="0e64ad73e4c8583d55755aadaf20a23357730788934bfe760a0566ec0a66ef90" exitCode=0 Jun 08 12:27:58.943474 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.943179 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" event={"ID":"3399c92c-e233-4003-88cc-585c696dd23f","Type":"ContainerDied","Data":"0e64ad73e4c8583d55755aadaf20a23357730788934bfe760a0566ec0a66ef90"} Jun 08 12:27:58.944855 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.944831 2576 generic.go:356] "Generic (PLEG): container finished" podID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" containerID="57b417a1d633ebe7081c312db4ddfbd3c225b008283d84e796dcca41aa649212" exitCode=0 Jun 08 12:27:58.944969 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.944911 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" event={"ID":"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9","Type":"ContainerDied","Data":"57b417a1d633ebe7081c312db4ddfbd3c225b008283d84e796dcca41aa649212"} Jun 08 12:27:58.996999 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.996968 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:27:58.999647 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:58.999631 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:27:59.090841 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.090815 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-475wz\" (UniqueName: \"kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz\") pod \"3399c92c-e233-4003-88cc-585c696dd23f\" (UID: \"3399c92c-e233-4003-88cc-585c696dd23f\") " Jun 08 12:27:59.091010 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.090853 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjvhk\" (UniqueName: \"kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk\") pod \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " Jun 08 12:27:59.091010 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.090883 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata\") pod \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\" (UID: \"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9\") " Jun 08 12:27:59.093079 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.093051 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk" (OuterVolumeSpecName: "kube-api-access-cjvhk") pod "3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" (UID: "3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9"). InnerVolumeSpecName "kube-api-access-cjvhk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:27:59.093180 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.093143 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz" (OuterVolumeSpecName: "kube-api-access-475wz") pod "3399c92c-e233-4003-88cc-585c696dd23f" (UID: "3399c92c-e233-4003-88cc-585c696dd23f"). InnerVolumeSpecName "kube-api-access-475wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:27:59.122582 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.122549 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" (UID: "3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:27:59.191686 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.191646 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-475wz\" (UniqueName: \"kubernetes.io/projected/3399c92c-e233-4003-88cc-585c696dd23f-kube-api-access-475wz\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:27:59.191686 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.191680 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjvhk\" (UniqueName: \"kubernetes.io/projected/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-kube-api-access-cjvhk\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:27:59.191686 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.191691 2576 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9-postgresdata\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:27:59.951108 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.951080 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" Jun 08 12:27:59.951565 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.951080 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-tqvwp" event={"ID":"3399c92c-e233-4003-88cc-585c696dd23f","Type":"ContainerDied","Data":"b16171a6cdad7e8b21653596d1ae1ee7b843baaba202bfa119b3bcaff3313943"} Jun 08 12:27:59.951565 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.951220 2576 scope.go:117] "RemoveContainer" containerID="0e64ad73e4c8583d55755aadaf20a23357730788934bfe760a0566ec0a66ef90" Jun 08 12:27:59.952471 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.952365 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" Jun 08 12:27:59.952471 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.952401 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-dsflg" event={"ID":"3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9","Type":"ContainerDied","Data":"ac4cd414aeefbb3f26ad4aae6aaad06692747d53eaa20f8886bc6a841cf17ec6"} Jun 08 12:27:59.960153 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.960133 2576 scope.go:117] "RemoveContainer" containerID="57b417a1d633ebe7081c312db4ddfbd3c225b008283d84e796dcca41aa649212" Jun 08 12:27:59.975010 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.974991 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:27:59.979472 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.979451 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-tqvwp"] Jun 08 12:27:59.990881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.990864 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:27:59.995055 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:27:59.995038 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-dsflg"] Jun 08 12:28:00.346251 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:00.346219 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" path="/var/lib/kubelet/pods/3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9/volumes" Jun 08 12:28:00.346544 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:00.346530 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3399c92c-e233-4003-88cc-585c696dd23f" path="/var/lib/kubelet/pods/3399c92c-e233-4003-88cc-585c696dd23f/volumes" Jun 08 12:28:01.709307 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:01.709258 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jun 08 12:28:03.442113 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:03.442070 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jun 08 12:28:11.709695 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:11.709652 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jun 08 12:28:13.442648 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:13.442605 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jun 08 12:28:21.709786 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:21.709747 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" probeResult="failure" output="Get \"https://10.132.0.26:6567/health\": dial tcp 10.132.0.26:6567: connect: connection refused" Jun 08 12:28:21.710136 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:21.709865 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:28:23.442292 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:23.442249 2576 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" probeResult="failure" output="Get \"https://10.132.0.27:6567/health\": dial tcp 10.132.0.27:6567: connect: connection refused" Jun 08 12:28:23.442685 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:23.442377 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:28:27.054233 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.054204 2576 generic.go:356] "Generic (PLEG): container finished" podID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerID="68ce74e02f8d4bb74fb7161d89fb5988b667fbd4965a75b496b53231a1b15df4" exitCode=137 Jun 08 12:28:27.054588 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.054277 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerDied","Data":"68ce74e02f8d4bb74fb7161d89fb5988b667fbd4965a75b496b53231a1b15df4"} Jun 08 12:28:27.644510 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.644484 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:28:27.668858 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.668838 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:28:27.808345 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808315 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls\") pod \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808362 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data\") pod \"d3313bd2-599f-4767-a022-1a9cb52f9055\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808396 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmsjw\" (UniqueName: \"kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw\") pod \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808426 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls\") pod \"d3313bd2-599f-4767-a022-1a9cb52f9055\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808442 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls\") pod \"d3313bd2-599f-4767-a022-1a9cb52f9055\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808458 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rl9gt\" (UniqueName: \"kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt\") pod \"d3313bd2-599f-4767-a022-1a9cb52f9055\" (UID: \"d3313bd2-599f-4767-a022-1a9cb52f9055\") " Jun 08 12:28:27.808505 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808478 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data\") pod \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " Jun 08 12:28:27.808844 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.808543 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls\") pod \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\" (UID: \"a8d5a183-e00d-4054-9d92-1473e9d4c42f\") " Jun 08 12:28:27.809285 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.809205 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data" (OuterVolumeSpecName: "feast-data") pod "a8d5a183-e00d-4054-9d92-1473e9d4c42f" (UID: "a8d5a183-e00d-4054-9d92-1473e9d4c42f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:28:27.810845 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.810811 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a8d5a183-e00d-4054-9d92-1473e9d4c42f" (UID: "a8d5a183-e00d-4054-9d92-1473e9d4c42f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:28:27.811317 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.811266 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt" (OuterVolumeSpecName: "kube-api-access-rl9gt") pod "d3313bd2-599f-4767-a022-1a9cb52f9055" (UID: "d3313bd2-599f-4767-a022-1a9cb52f9055"). InnerVolumeSpecName "kube-api-access-rl9gt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:28:27.811317 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.811278 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls" (OuterVolumeSpecName: "online-tls") pod "a8d5a183-e00d-4054-9d92-1473e9d4c42f" (UID: "a8d5a183-e00d-4054-9d92-1473e9d4c42f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:28:27.811317 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.811271 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls" (OuterVolumeSpecName: "online-tls") pod "d3313bd2-599f-4767-a022-1a9cb52f9055" (UID: "d3313bd2-599f-4767-a022-1a9cb52f9055"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:28:27.811317 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.811287 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d3313bd2-599f-4767-a022-1a9cb52f9055" (UID: "d3313bd2-599f-4767-a022-1a9cb52f9055"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 12:28:27.811491 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.811350 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw" (OuterVolumeSpecName: "kube-api-access-kmsjw") pod "a8d5a183-e00d-4054-9d92-1473e9d4c42f" (UID: "a8d5a183-e00d-4054-9d92-1473e9d4c42f"). InnerVolumeSpecName "kube-api-access-kmsjw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:28:27.817755 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.817735 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data" (OuterVolumeSpecName: "feast-data") pod "d3313bd2-599f-4767-a022-1a9cb52f9055" (UID: "d3313bd2-599f-4767-a022-1a9cb52f9055"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:28:27.909378 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909340 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3313bd2-599f-4767-a022-1a9cb52f9055-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909378 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909370 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmsjw\" (UniqueName: \"kubernetes.io/projected/a8d5a183-e00d-4054-9d92-1473e9d4c42f-kube-api-access-kmsjw\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909378 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909380 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909388 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3313bd2-599f-4767-a022-1a9cb52f9055-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909396 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl9gt\" (UniqueName: \"kubernetes.io/projected/d3313bd2-599f-4767-a022-1a9cb52f9055-kube-api-access-rl9gt\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909405 2576 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a8d5a183-e00d-4054-9d92-1473e9d4c42f-feast-data\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909413 2576 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-online-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:27.909617 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:27.909420 2576 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a8d5a183-e00d-4054-9d92-1473e9d4c42f-registry-tls\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:28:28.059891 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.059806 2576 generic.go:356] "Generic (PLEG): container finished" podID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerID="13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877" exitCode=137 Jun 08 12:28:28.059891 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.059872 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" Jun 08 12:28:28.060369 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.059887 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerDied","Data":"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877"} Jun 08 12:28:28.060369 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.059947 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2" event={"ID":"a8d5a183-e00d-4054-9d92-1473e9d4c42f","Type":"ContainerDied","Data":"96dd8b4a63ee99ecf4e8c1dcebc524107cecdb865fc3fb343fe3a3532d24d241"} Jun 08 12:28:28.060369 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.059975 2576 scope.go:117] "RemoveContainer" containerID="8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4" Jun 08 12:28:28.065728 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.062748 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" event={"ID":"d3313bd2-599f-4767-a022-1a9cb52f9055","Type":"ContainerDied","Data":"ef3d7a4e62d83fc320336eeb48d594d7567137597f1b5ad895e44d9adcb683ef"} Jun 08 12:28:28.065728 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.062887 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq" Jun 08 12:28:28.073648 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.073392 2576 scope.go:117] "RemoveContainer" containerID="13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877" Jun 08 12:28:28.081943 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.081926 2576 scope.go:117] "RemoveContainer" containerID="c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b" Jun 08 12:28:28.088759 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.088704 2576 scope.go:117] "RemoveContainer" containerID="7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6" Jun 08 12:28:28.090607 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.090587 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:28:28.095881 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.095852 2576 scope.go:117] "RemoveContainer" containerID="8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4" Jun 08 12:28:28.096152 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:28:28.096136 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4\": container with ID starting with 8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4 not found: ID does not exist" containerID="8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4" Jun 08 12:28:28.096218 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096158 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4"} err="failed to get container status \"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4\": rpc error: code = NotFound desc = could not find container \"8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4\": container with ID starting with 8543bd52b2e67413fd4f1e8b24a77489f5df1fd780b93b9d5e764a61f7d58ab4 not found: ID does not exist" Jun 08 12:28:28.096218 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096175 2576 scope.go:117] "RemoveContainer" containerID="13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877" Jun 08 12:28:28.096403 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:28:28.096385 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877\": container with ID starting with 13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877 not found: ID does not exist" containerID="13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877" Jun 08 12:28:28.096441 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096410 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877"} err="failed to get container status \"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877\": rpc error: code = NotFound desc = could not find container \"13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877\": container with ID starting with 13ff2129d3fd869e03042e7ae36f61718d22e6068898213bd83d8efbaa2f2877 not found: ID does not exist" Jun 08 12:28:28.096441 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096425 2576 scope.go:117] "RemoveContainer" containerID="c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b" Jun 08 12:28:28.096683 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:28:28.096666 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b\": container with ID starting with c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b not found: ID does not exist" containerID="c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b" Jun 08 12:28:28.096744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096686 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b"} err="failed to get container status \"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b\": rpc error: code = NotFound desc = could not find container \"c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b\": container with ID starting with c862d295c58b8a497bbdd951db4c041ae574ec83c398941c3d20c736e4a6c11b not found: ID does not exist" Jun 08 12:28:28.096744 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096699 2576 scope.go:117] "RemoveContainer" containerID="7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6" Jun 08 12:28:28.096940 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:28:28.096922 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6\": container with ID starting with 7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6 not found: ID does not exist" containerID="7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6" Jun 08 12:28:28.097001 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096949 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6"} err="failed to get container status \"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6\": rpc error: code = NotFound desc = could not find container \"7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6\": container with ID starting with 7adf8445251c7effc1296f3cc1b4b7d913662281c331f777004232e29a22e3f6 not found: ID does not exist" Jun 08 12:28:28.097001 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.096971 2576 scope.go:117] "RemoveContainer" containerID="0bd6e35548a1c114145268784cca543e45922d3ac29ecbf985bb9a21a570c498" Jun 08 12:28:28.097102 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.097034 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6dbc7c5698-m9ks2"] Jun 08 12:28:28.104336 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.104320 2576 scope.go:117] "RemoveContainer" containerID="68ce74e02f8d4bb74fb7161d89fb5988b667fbd4965a75b496b53231a1b15df4" Jun 08 12:28:28.111353 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.111334 2576 scope.go:117] "RemoveContainer" containerID="5c15bbb5177cc11d95afe6bd8152472f0747d0e6ce39c23e490904b90eee14ce" Jun 08 12:28:28.113575 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.113552 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:28:28.118408 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.118388 2576 scope.go:117] "RemoveContainer" containerID="d64d62b911eb8e3257e8a7d9b0726b600bc51b61a43ca5ed45fbe01e9fa52cd2" Jun 08 12:28:28.127905 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.127881 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-66b4d7c4d9-fxwlq"] Jun 08 12:28:28.345714 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.345641 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" path="/var/lib/kubelet/pods/a8d5a183-e00d-4054-9d92-1473e9d4c42f/volumes" Jun 08 12:28:28.346273 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:28:28.346252 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" path="/var/lib/kubelet/pods/d3313bd2-599f-4767-a022-1a9cb52f9055/volumes" Jun 08 12:38:50.175080 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175041 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fd6vk/must-gather-gmgl6"] Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175306 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="feast-apply" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175317 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="feast-apply" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175329 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8290a8ca-37af-4eca-9006-3a50fbc17a06" containerName="manager" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175335 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8290a8ca-37af-4eca-9006-3a50fbc17a06" containerName="manager" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175343 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175348 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175355 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3399c92c-e233-4003-88cc-585c696dd23f" containerName="redis" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175360 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3399c92c-e233-4003-88cc-585c696dd23f" containerName="redis" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175365 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" containerName="postgres" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175370 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" containerName="postgres" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175378 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175382 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175388 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175393 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175399 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="feast-init" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175406 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="feast-init" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175413 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="feast-apply" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175418 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="feast-apply" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175425 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="feast-init" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175430 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="feast-init" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175435 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175440 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175479 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="registry" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175488 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3313bd2-599f-4767-a022-1a9cb52f9055" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175495 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8290a8ca-37af-4eca-9006-3a50fbc17a06" containerName="manager" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175500 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="online" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175506 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3399c92c-e233-4003-88cc-585c696dd23f" containerName="redis" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175512 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="3103f6e0-9acb-4d24-aa82-ac6a0c8f6ac9" containerName="postgres" Jun 08 12:38:50.176462 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.175531 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8d5a183-e00d-4054-9d92-1473e9d4c42f" containerName="registry" Jun 08 12:38:50.177348 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.177196 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.181140 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.181118 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd6vk\"/\"openshift-service-ca.crt\"" Jun 08 12:38:50.182038 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.182024 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fd6vk\"/\"default-dockercfg-6gh24\"" Jun 08 12:38:50.182120 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.182032 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fd6vk\"/\"kube-root-ca.crt\"" Jun 08 12:38:50.191916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.191886 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd6vk/must-gather-gmgl6"] Jun 08 12:38:50.280181 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.280143 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.280376 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.280195 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r2d4\" (UniqueName: \"kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.380938 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.380895 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.381128 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.380953 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r2d4\" (UniqueName: \"kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.381260 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.381239 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.391555 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.391502 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r2d4\" (UniqueName: \"kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4\") pod \"must-gather-gmgl6\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.486088 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.485975 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:38:50.623530 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.623482 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fd6vk/must-gather-gmgl6"] Jun 08 12:38:50.629140 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:38:50.629083 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod154a4f54_5253_4238_931d_6463a6d6aa32.slice/crio-51330d41470e64d2447571e93d224ce78d36a146eced5485732614c6c00961f8 WatchSource:0}: Error finding container 51330d41470e64d2447571e93d224ce78d36a146eced5485732614c6c00961f8: Status 404 returned error can't find the container with id 51330d41470e64d2447571e93d224ce78d36a146eced5485732614c6c00961f8 Jun 08 12:38:50.630707 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:50.630689 2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 12:38:51.135064 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:51.135025 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" event={"ID":"154a4f54-5253-4238-931d-6463a6d6aa32","Type":"ContainerStarted","Data":"51330d41470e64d2447571e93d224ce78d36a146eced5485732614c6c00961f8"} Jun 08 12:38:57.169223 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:57.169181 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" event={"ID":"154a4f54-5253-4238-931d-6463a6d6aa32","Type":"ContainerStarted","Data":"82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef"} Jun 08 12:38:57.169766 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:57.169236 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" event={"ID":"154a4f54-5253-4238-931d-6463a6d6aa32","Type":"ContainerStarted","Data":"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e"} Jun 08 12:38:57.190877 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:38:57.190825 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" podStartSLOduration=1.172037341 podStartE2EDuration="7.190808096s" totalImagesPullingTime="6.018770755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 12:38:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:38:57.190454856 +0000 UTC m=+1905.300711082" watchObservedRunningTime="2026-06-08 12:38:57.190808096 +0000 UTC m=+1905.301064321" Jun 08 12:39:07.204258 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:07.204223 2576 generic.go:356] "Generic (PLEG): container finished" podID="154a4f54-5253-4238-931d-6463a6d6aa32" containerID="430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e" exitCode=0 Jun 08 12:39:07.204690 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:07.204283 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" event={"ID":"154a4f54-5253-4238-931d-6463a6d6aa32","Type":"ContainerDied","Data":"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e"} Jun 08 12:39:07.204690 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:07.204603 2576 scope.go:117] "RemoveContainer" containerID="430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e" Jun 08 12:39:07.619227 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:07.619199 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fd6vk_must-gather-gmgl6_154a4f54-5253-4238-931d-6463a6d6aa32/gather/0.log" Jun 08 12:39:10.985641 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:10.985606 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wklwq_03c699ff-b0c6-44af-bd0a-8b8679dbf4c9/global-pull-secret-syncer/0.log" Jun 08 12:39:11.120715 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:11.120674 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jgc27_a7cfca38-7b21-4e52-8d90-79f59e53ba74/konnectivity-agent/0.log" Jun 08 12:39:11.188595 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:11.188562 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-216.ec2.internal_b7f7b943bea851ddb58741f3f1e3310a/haproxy/0.log" Jun 08 12:39:12.991279 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:12.991238 2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-fd6vk/must-gather-gmgl6"] Jun 08 12:39:12.991732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:12.991431 2576 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="copy" containerID="cri-o://82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef" gracePeriod=2 Jun 08 12:39:12.993325 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:12.993296 2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-fd6vk/must-gather-gmgl6"] Jun 08 12:39:12.993699 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:12.993660 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gmgl6\" is forbidden: User \"system:node:ip-10-0-128-216.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fd6vk\": no relationship found between node 'ip-10-0-128-216.ec2.internal' and this object" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:13.223407 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.223381 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fd6vk_must-gather-gmgl6_154a4f54-5253-4238-931d-6463a6d6aa32/copy/0.log" Jun 08 12:39:13.224851 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.224830 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:13.226868 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.226850 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fd6vk_must-gather-gmgl6_154a4f54-5253-4238-931d-6463a6d6aa32/copy/0.log" Jun 08 12:39:13.227218 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.227196 2576 generic.go:356] "Generic (PLEG): container finished" podID="154a4f54-5253-4238-931d-6463a6d6aa32" containerID="82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef" exitCode=143 Jun 08 12:39:13.227288 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.227241 2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:13.227288 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.227270 2576 scope.go:117] "RemoveContainer" containerID="82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef" Jun 08 12:39:13.227353 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:13.227290 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gmgl6\" is forbidden: User \"system:node:ip-10-0-128-216.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fd6vk\": no relationship found between node 'ip-10-0-128-216.ec2.internal' and this object" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:13.229752 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:13.229729 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gmgl6\" is forbidden: User \"system:node:ip-10-0-128-216.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fd6vk\": no relationship found between node 'ip-10-0-128-216.ec2.internal' and this object" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:13.234432 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.234410 2576 scope.go:117] "RemoveContainer" containerID="430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e" Jun 08 12:39:13.246621 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.246596 2576 scope.go:117] "RemoveContainer" containerID="82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef" Jun 08 12:39:13.246946 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:13.246929 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef\": container with ID starting with 82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef not found: ID does not exist" containerID="82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef" Jun 08 12:39:13.246993 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.246956 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef"} err="failed to get container status \"82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef\": rpc error: code = NotFound desc = could not find container \"82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef\": container with ID starting with 82d552d6cb4eae3f69b5b8b2b0044e09d35979f69659c25b261459860f0222ef not found: ID does not exist" Jun 08 12:39:13.246993 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.246973 2576 scope.go:117] "RemoveContainer" containerID="430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e" Jun 08 12:39:13.247197 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:13.247178 2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e\": container with ID starting with 430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e not found: ID does not exist" containerID="430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e" Jun 08 12:39:13.247245 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.247202 2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e"} err="failed to get container status \"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e\": rpc error: code = NotFound desc = could not find container \"430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e\": container with ID starting with 430adcfdc841eae54b45504e6f13000da58d534d0ba772271c909ac00d17283e not found: ID does not exist" Jun 08 12:39:13.354984 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.354947 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r2d4\" (UniqueName: \"kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4\") pod \"154a4f54-5253-4238-931d-6463a6d6aa32\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " Jun 08 12:39:13.355121 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.355067 2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output\") pod \"154a4f54-5253-4238-931d-6463a6d6aa32\" (UID: \"154a4f54-5253-4238-931d-6463a6d6aa32\") " Jun 08 12:39:13.355452 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.355430 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "154a4f54-5253-4238-931d-6463a6d6aa32" (UID: "154a4f54-5253-4238-931d-6463a6d6aa32"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 12:39:13.357213 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.357183 2576 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4" (OuterVolumeSpecName: "kube-api-access-6r2d4") pod "154a4f54-5253-4238-931d-6463a6d6aa32" (UID: "154a4f54-5253-4238-931d-6463a6d6aa32"). InnerVolumeSpecName "kube-api-access-6r2d4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 12:39:13.455916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.455860 2576 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/154a4f54-5253-4238-931d-6463a6d6aa32-must-gather-output\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:39:13.455916 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:13.455906 2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r2d4\" (UniqueName: \"kubernetes.io/projected/154a4f54-5253-4238-931d-6463a6d6aa32-kube-api-access-6r2d4\") on node \"ip-10-0-128-216.ec2.internal\" DevicePath \"\"" Jun 08 12:39:13.537943 ip-10-0-128-216 kubenswrapper[2576]: E0608 12:39:13.537914 2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-gmgl6\" is forbidden: User \"system:node:ip-10-0-128-216.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fd6vk\": no relationship found between node 'ip-10-0-128-216.ec2.internal' and this object" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" pod="openshift-must-gather-fd6vk/must-gather-gmgl6" Jun 08 12:39:14.351717 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:14.349857 2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" path="/var/lib/kubelet/pods/154a4f54-5253-4238-931d-6463a6d6aa32/volumes" Jun 08 12:39:15.276011 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.275982 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk4hz_5296e584-d40e-4cbb-aa50-c581fceed528/kube-state-metrics/0.log" Jun 08 12:39:15.306197 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.306168 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk4hz_5296e584-d40e-4cbb-aa50-c581fceed528/kube-rbac-proxy-main/0.log" Jun 08 12:39:15.332716 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.332689 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk4hz_5296e584-d40e-4cbb-aa50-c581fceed528/kube-rbac-proxy-self/0.log" Jun 08 12:39:15.534048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.533970 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl44h_94bd54c1-8bae-4f80-ac32-72ce49b79474/node-exporter/0.log" Jun 08 12:39:15.566472 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.566445 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl44h_94bd54c1-8bae-4f80-ac32-72ce49b79474/kube-rbac-proxy/0.log" Jun 08 12:39:15.592368 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:15.592342 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl44h_94bd54c1-8bae-4f80-ac32-72ce49b79474/init-textfile/0.log" Jun 08 12:39:16.106939 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:16.106909 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5b6b8f594-vw9fz_22faebf6-91aa-4907-8dbd-4512eb96777e/prometheus-operator-admission-webhook/0.log" Jun 08 12:39:16.149321 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:16.149295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f8649bc-jprlp_b5edfb3a-f9ed-43c1-b92d-07c66bd37912/telemeter-client/0.log" Jun 08 12:39:16.189214 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:16.189184 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f8649bc-jprlp_b5edfb3a-f9ed-43c1-b92d-07c66bd37912/reload/0.log" Jun 08 12:39:16.224219 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:16.224188 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f8649bc-jprlp_b5edfb3a-f9ed-43c1-b92d-07c66bd37912/kube-rbac-proxy/0.log" Jun 08 12:39:18.491147 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.491114 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-lp8ct_eeb625a7-05b2-4f76-ae7f-098c104a72c6/download-server/0.log" Jun 08 12:39:18.510038 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510004 2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz"] Jun 08 12:39:18.510254 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510244 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="copy" Jun 08 12:39:18.510299 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510256 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="copy" Jun 08 12:39:18.510299 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510277 2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="gather" Jun 08 12:39:18.510299 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510282 2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="gather" Jun 08 12:39:18.510386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510322 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="gather" Jun 08 12:39:18.510386 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.510331 2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="154a4f54-5253-4238-931d-6463a6d6aa32" containerName="copy" Jun 08 12:39:18.513702 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.513670 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.516082 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.516055 2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jdm9r\"/\"default-dockercfg-vqvdl\"" Jun 08 12:39:18.516260 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.516241 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdm9r\"/\"openshift-service-ca.crt\"" Jun 08 12:39:18.517202 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.517176 2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jdm9r\"/\"kube-root-ca.crt\"" Jun 08 12:39:18.525625 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.525597 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz"] Jun 08 12:39:18.690029 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.689988 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56xn\" (UniqueName: \"kubernetes.io/projected/130b1255-55fe-4829-8f3d-3b0772d6a775-kube-api-access-d56xn\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.690029 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.690027 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-proc\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.690248 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.690122 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-sys\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.690248 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.690155 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-lib-modules\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.690248 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.690188 2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-podres\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790561 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790501 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-sys\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790569 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-lib-modules\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790597 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-podres\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790621 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56xn\" (UniqueName: \"kubernetes.io/projected/130b1255-55fe-4829-8f3d-3b0772d6a775-kube-api-access-d56xn\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790635 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-sys\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790639 2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-proc\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.790732 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790694 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-proc\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.791020 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790760 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-lib-modules\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.791020 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.790761 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/130b1255-55fe-4829-8f3d-3b0772d6a775-podres\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.799344 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.799304 2576 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56xn\" (UniqueName: \"kubernetes.io/projected/130b1255-55fe-4829-8f3d-3b0772d6a775-kube-api-access-d56xn\") pod \"perf-node-gather-daemonset-rncgz\" (UID: \"130b1255-55fe-4829-8f3d-3b0772d6a775\") " pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.823288 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.823249 2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:18.954479 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:18.954439 2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz"] Jun 08 12:39:18.957666 ip-10-0-128-216 kubenswrapper[2576]: W0608 12:39:18.957633 2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod130b1255_55fe_4829_8f3d_3b0772d6a775.slice/crio-e8cab4b7bed193503d5bbed4d022daf2d670654781337987bc12a367959797fa WatchSource:0}: Error finding container e8cab4b7bed193503d5bbed4d022daf2d670654781337987bc12a367959797fa: Status 404 returned error can't find the container with id e8cab4b7bed193503d5bbed4d022daf2d670654781337987bc12a367959797fa Jun 08 12:39:19.251048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.250956 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" event={"ID":"130b1255-55fe-4829-8f3d-3b0772d6a775","Type":"ContainerStarted","Data":"5fcd097d25997385f45b0db856c4e006b17e145a67cb23e62e9383f68fc7e97c"} Jun 08 12:39:19.251048 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.251014 2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" event={"ID":"130b1255-55fe-4829-8f3d-3b0772d6a775","Type":"ContainerStarted","Data":"e8cab4b7bed193503d5bbed4d022daf2d670654781337987bc12a367959797fa"} Jun 08 12:39:19.251239 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.251051 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:19.270159 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.270055 2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" podStartSLOduration=1.2700337529999999 podStartE2EDuration="1.270033753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 12:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 12:39:19.268837719 +0000 UTC m=+1927.379093942" watchObservedRunningTime="2026-06-08 12:39:19.270033753 +0000 UTC m=+1927.380289974" Jun 08 12:39:19.847681 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.847647 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bv5jn_a9ef63cf-e8c1-427a-8376-18b4dc9d6cce/dns/0.log" Jun 08 12:39:19.871325 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.871295 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bv5jn_a9ef63cf-e8c1-427a-8376-18b4dc9d6cce/kube-rbac-proxy/0.log" Jun 08 12:39:19.979599 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:19.979565 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-mg5dx_502a1100-b449-457e-9de5-7664cd9e5437/dns-node-resolver/0.log" Jun 08 12:39:20.430982 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:20.430942 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-64znz_77147794-d0d1-404f-a3f1-ea0d2d9627c9/node-ca/0.log" Jun 08 12:39:21.518485 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:21.518451 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cmk7n_22a1a1d0-cbd6-4d4b-8ef3-8b6f16791af2/serve-healthcheck-canary/0.log" Jun 08 12:39:22.012726 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:22.012699 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nclf6_272c8051-3716-48df-919b-d975fb099487/kube-rbac-proxy/0.log" Jun 08 12:39:22.035158 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:22.035132 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nclf6_272c8051-3716-48df-919b-d975fb099487/exporter/0.log" Jun 08 12:39:22.058850 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:22.058821 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nclf6_272c8051-3716-48df-919b-d975fb099487/extractor/0.log" Jun 08 12:39:25.268025 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:25.267995 2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jdm9r/perf-node-gather-daemonset-rncgz" Jun 08 12:39:28.398513 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.398484 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/kube-multus-additional-cni-plugins/0.log" Jun 08 12:39:28.436629 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.436585 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/egress-router-binary-copy/0.log" Jun 08 12:39:28.471534 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.471491 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/cni-plugins/0.log" Jun 08 12:39:28.522473 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.522399 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/bond-cni-plugin/0.log" Jun 08 12:39:28.556153 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.556119 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/routeoverride-cni/0.log" Jun 08 12:39:28.586090 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.586066 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/whereabouts-cni-bincopy/0.log" Jun 08 12:39:28.615699 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.615665 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-grllw_f516abd9-ddbb-4687-86a2-74ffe4d295b9/whereabouts-cni/0.log" Jun 08 12:39:28.722889 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.722853 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tqk8p_6d0ab198-453d-4179-b1c1-78eca2c0fa8d/kube-multus/0.log" Jun 08 12:39:28.910760 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.910728 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ztpgp_b1e44096-d501-40a5-bd8a-33e19aa19e20/network-metrics-daemon/0.log" Jun 08 12:39:28.955919 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:28.955867 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ztpgp_b1e44096-d501-40a5-bd8a-33e19aa19e20/kube-rbac-proxy/0.log" Jun 08 12:39:29.785642 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.785611 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/ovn-controller/0.log" Jun 08 12:39:29.811458 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.811426 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/ovn-acl-logging/0.log" Jun 08 12:39:29.838778 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.838678 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/kube-rbac-proxy-node/0.log" Jun 08 12:39:29.864594 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.864560 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 12:39:29.886045 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.886005 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/northd/0.log" Jun 08 12:39:29.911551 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.911507 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/nbdb/0.log" Jun 08 12:39:29.940586 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.940554 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/sbdb/0.log" Jun 08 12:39:29.986805 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:29.986753 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g5z25_16af954c-1c38-473e-9697-211ec646d981/ovnkube-controller/0.log" Jun 08 12:39:31.474141 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:31.474106 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-z2wzj_b7f15cc3-634e-4fe3-bf78-d5edc451a6aa/network-check-target-container/0.log" Jun 08 12:39:32.514686 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:32.514655 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-pgtkw_8fe0067c-3d19-4070-8915-df87bb6cc277/iptables-alerter/0.log" Jun 08 12:39:33.242673 ip-10-0-128-216 kubenswrapper[2576]: I0608 12:39:33.242629 2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-xlpvx_c5c65557-a24d-4760-a660-fcc0acfb6dd7/tuned/0.log"